basic-text

This repository hosts a project to define a text format called [Basic Text], a subset of Unicode that focuses on text: it excludes unprintable control characters, characters which depend on out-of-band information to interpret, non-characters, deprecated characters, and more, while aiming to preserve everything of practical use to plain text and formats built on top of it. See the [Book] for more information.

It also includes a Rust crate which aims to implement the Basic Text format, providing several utilities:

• [TextString] and [TextStr] are similar to the standard library's [String] and [str], but use the Basic Text string format, along with a [text!("...") macro] for Basic Text string literals.

• [TextReader] and [TextWriter] are input and output streams which use the Basic Text stream format. On input, content is converted in a way which is lossy with respect to the original bytestream. Output uses the "strict" conversion method, in which invalid content is diagnosed with errors.

• [BufReadText], an extension trait that adds [text_lines] and [text_lines_lossy] to [BufRead] implementations for reading lines from an input stream as BasicText strings.

• [TextDuplexer] is a [Duplex] for reading and writing on an interactive stream using Basic Text.

The code here is usable, and implements everything in the [Basic Text] spec, but is not very mature or optimized yet.