ratatu-image

GitHub CI Status

Showcase:

Recording

Image widgets for [Ratatui]

⚠️ THIS CRATE IS EXPERIMENTAL

Render images with graphics protocols in the terminal with [Ratatui].

```rust struct App { image: Box, }

fn main() -> Result<(), Box> { let fontsize = (7, 16); // Or use Picker::fromtermios, or let user provide it. let mut picker = Picker::new( fontsize, BackendType::Sixel, None, )?; let dynimg = image::io::Reader::open("./assets/Ada.png")?.decode()?; let image = picker.newstaticfit(dyn_img, Rect::new(0, 0, 30, 20), Resize::Fit)?; let mut app = App { image };

let backend = TestBackend::new(80, 30);
let mut terminal = Terminal::new(backend)?;

// loop:
terminal.draw(|f| ui(f, &mut app))?;

Ok(())

}

fn ui(f: &mut Frame, app: &mut App) { let image = FixedImage::new(app.image.asref()); f.renderwidget(image, f.size()); } ```

TUIs

TUI application revolve around columns and rows of text naturally without the need of any notions of pixel sizes. [Ratatui] is based on "immediate rendering with intermediate buffers".

At each frame, widgets are constructed and rendered into some character buffer, and any changes from respect to the last frame are then diffed and written to the terminal screen.

Terminal graphic protocols

Some protocols allow to output image data to terminals that support it.

The [Sixel] protocol mechanism is, in a nutshell, just printing an escape sequence. The image will be "dumped" at the cursor position, and the implementation may add enough carriage returns to scroll the output.

Problem

Simply "dumping" an image into a [Ratatui] buffer is not enough. At best, the buffer diff might not overwrite any characters that are covered by the image in some instances, but the diff might change at any time due to screen/area resizing or simply other widget's contents changing. Then the graphics would inmediately get overwritten by the underlying character data.

Solution

First it is necessary to suppress the covered character cells' rendering, which is addressed in a [Ratatui PR for cell skipping].

Second it is then necessary to get the image's size in columns and rows, which is done by querying the terminal for it's pixel size and dividing by columns/rows to get the font size in pixels. Currently this is implemented with rustix::termios, but this is subject to change for a [Ratatui PR for getting window size].

Implementation

The images are always resized so that they fit their nearest rectangle in columns/rows. This is so that the image shall be drawn in the same "render pass" as all surrounding text, and cells under the area of the image skip the draw on the ratatui buffer level, so there is no way to "clear" previous drawn text. This would leave artifacts around the image's right and bottom borders.

Example

See the [crate::picker::Picker] helper and examples/demo.

Current version: 0.1.1

Sixel compatibility and QA:

Terminal | Fixed | Resize | Notes -----------|-------|--------|------- Xterm | ✔️ | ✔️ | Foot | ✔️ | ✔️ | kitty | ✔️ | ✔️ | Alacritty | ✔️ | ❌ | with sixel patch, never clears graphics. iTerm2 | ❌ | ❌ | Unimplemented, has a protocolo similar to sixel konsole | ❌ | ❌ | Does not clear graphics unless cells have a background style Contour | ❌ | ❌ | Text over graphics Wezterm | ❌ | ❌ | Buggy ctx | ❌ | ❌ | Buggy Blackbox | ❔ | ❔ | Untested

Latest Xterm testing screenshot:
Testing screenshot

Halfblocks should work in all terminals.

Comparison:

License: MIT