A cross-platform library for fast and safe memory-mapped IO in Rust
Take a look at the Documentation for details!
This library defines a convenient API for reading and writing to files
using the hosts virtual memory system. The design of the API strives to
both minimize the frequency of mapping system calls while still retaining
safe access. Critically, it never attempts the own the File object used
for mapping. That is, it never clones it or in any way retains it. While
this has some implications for the API (i.e. [.flush()]), it cannot cause
bugs outside of this library through File's leaky abstraction when cloned
and then closed.
The [Map] and [MapMut] types are primary means for allocating virtual
memory regions, both for a file and anonymously. Generally, the
[Map::with_options()] and [MapMut::with_options()] are used to specify
the mapping requirements. See [Options] for more information.
The [MapMut] type maintains interior mutability for the mapped memory,
while the [Map] is read-only. However, it is possible to convert between
these types ([.into_map_mut()] and [.into_map()]) assuming the proper
[Options] are specified.
Additionally, a variety of buffer implementations are provided in the
[vmap::io] module. The [Ring] and [InfiniteRing] use circular memory
address allocations using cross-platform optimizations to minimize excess
resources where possible. The [BufReader] and [BufWriter] implement
buffered I/O using a [Ring] as a backing layer.
```rust use vmap::Map; use std::{fs, str};
let path = "example";
// Write some test data fs::write(&path, b"this is a test")?;
// Map the first 4 bytes let (map, file) = Map::withoptions().len(4).open(&path)?; asserteq!(Ok("this"), str::from_utf8(&map[..]));
// Reuse the file to map a different region let map = Map::withoptions().offset(10).len(4).map(&file)?; asserteq!(Ok("test"), str::from_utf8(&map[..])); ```
If opened properly, the [Map] can be moved into a [MapMut] and modifications
to the underlying file can be performed:
```rust use vmap::Map; use std::{fs, str};
let path = "example";
// Write some test data fs::write(&path, b"this is a test")?;
// Open with write permissions so the Map can be converted into a MapMut let (map, file) = Map::withoptions().write().len(14).open(&path)?; asserteq!(Ok("this is a test"), str::from_utf8(&map[..]));
// Move the Map into a MapMut // ... we could have started with MapMut::withoptions() let mut map = map.intomapmut()?; map[..4].clonefrom_slice(b"that");
// Flush the changes to disk synchronously map.flush(&file, Flush::Sync)?;
// Move the MapMut back into a Map let map = map.intomap()?; asserteq!(Ok("that is a test"), str::from_utf8(&map[..])); ```
The [vmap] library contains a [Ring] that constructs a circular memory
allocation where values can wrap from around from the end of the buffer back
to the beginning with sequential memory addresses. The [InfiniteRing] is
similar, however it allows writes to overwrite reads.
```rust use vmap::io::{Ring, SeqWrite}; use std::io::{BufRead, Read, Write};
let mut buf = Ring::new(4000).unwrap(); let mut i = 1;
// Fill up the buffer with lines. while buf.write_len() > 20 { write!(&mut buf, "this is test line {}\n", i)?; i += 1; }
// No more space is available. assert!(write!(&mut buf, "this is test line {}\n", i).is_err());
let mut line = String::new();
// Read the first line written. let len = buf.readline(&mut line)?; asserteq!(line, "this is test line 1\n");
line.clear();
// Read the second line written. let len = buf.readline(&mut line)?; asserteq!(line, "this is test line 2\n");
// Now there is enough space to write more. write!(&mut buf, "this is test line {}\n", i)?; ```