WebNative FileSystem (WNFS)

#

This is a Rust implementation of the WebNative FileSystem (WNFS) specification. WNFS is a versioned content-addressable distributed filesystem with private and public sub systems. The private filesystem is encrypted so that only users with the right keys can access its contents. It is designed to prevent inferring metadata like the structure of the file tree. The other part of the WNFS filesystem is a simpler public filesystem that is not encrypted and can be accessed by anyone with the right address.

WNFS also features collaborative editing of file trees, where multiple users can edit the same tree at the same time.

WNFS file trees can serialize and be deserialized from IPLD graphs with an extensible metadata section. This allows WNFS to be understood by other IPLD-based tools and systems.

Usage

WNFS does not have an opinion on where you want to persist your content or the file tree. Instead, the API expects any object that implements the async BlockStore interface. This implementation also defers system-level operations to the user; requiring that operations like time and random number generation be passed in from the interface. This makes for a clean wasm interface that works everywhere.

Let's see an example of working with a public directory. Here we are going to use the memory-based blockstore provided by the library.

```rust use wnfs::{MemoryBlockStore, PublicDirectory, PublicOpResult};

use chrono::Utc;

use std::rc::Rc;

[async_std::main]

async fn main() { // Create a new public directory. let dir = Rc::new(PublicDirectory::new(Utc::now()));

// Create a memory-based blockstore.
let store = &mut MemoryBlockStore::default();

// Add a /pictures/cats subdirectory.
let PublicOpResult { root_dir, .. } = dir
    .mkdir(&["pictures".into(), "cats".into()], Utc::now(), store)
    .await
    .unwrap();

// Store the the file tree in the memory blockstore.
root_dir.store(store).await.unwrap();

// Print root directory.
println!("{:#?}", root_dir);

} ```

You may notice that we store the root_dir returned by the mkdir operation, not the dir we started with. That is because WNFS internal state is immutable and every operation potentially returns a new root directory. This allows us to track and rollback changes when needed. It also makes collaborative editing easier to implement and reason about. You can find more examples in the wnfs/examples/ folder. And there is a basic demo of the filesystem immutability here.

The private filesystem, on the other hand, is a bit more involved. Hash Array Mapped Trie (HAMT) is used as the intermediate format of private file tree before it is persisted to the blockstore. Our use of HAMTs obfuscate the file tree hierarchy.

```rust use wnfs::{ private::PrivateForest, MemoryBlockStore, Namefilter, PrivateDirectory, PrivateOpResult, };

use chrono::Utc; use rand::thread_rng;

use std::rc::Rc;

[async_std::main]

async fn main() { // Create a memory-based blockstore. let store = &mut MemoryBlockStore::default();

// A random number generator the private filesystem can use.
let rng = &mut thread_rng();

// Create private forest.
let forest = Rc::new(PrivateForest::new());

// Create a new private directory.
let dir = Rc::new(PrivateDirectory::new(
    Namefilter::default(),
    Utc::now(),
    rng,
));

// Add a file to /pictures/cats directory.
let PrivateOpResult { root_dir, forest, .. } = dir
    .mkdir(
        &["pictures".into(), "cats".into()],
        true,
        Utc::now(),
        forest,
        store,
        rng,
    )
    .await
    .unwrap();

// Add a file to /pictures/dogs/billie.jpg file.
let PrivateOpResult { root_dir, forest, .. } = root_dir
    .write(
        &["pictures".into(), "dogs".into(), "billie.jpeg".into()],
        true,
        Utc::now(),
        b"hello world".to_vec(),
        forest,
        store,
        rng,
    )
    .await
    .unwrap();

// List all files in /pictures directory.
let PrivateOpResult { result, .. } = root_dir
    .ls(&["pictures".into()], true, forest, store)
    .await
    .unwrap();

println!("Files in /pictures: {:#?}", result);

} ```

Namefilters are currently how we identify private node blocks in the filesystem. They have nice properties, one of which is the ability to check if one node belongs to another. This is necessary in a filesystem where metadata like hierarchy needs to be hidden from observing agents. One notable caveat with namefilters is that they can only reliably store information of a file tree 47 levels deep or less so there is a plan to replace them with other cryptographic accumlators in the near future.

Check the examples/ folder for more examples.