Web-RWKV

crates.io docs.rs

This is an implementation of the language model of RWKV in pure WebGPU.

Compile and Run

  1. Install Rust.
  2. Run cargo run --release --example gen to generate 100 tokens and measure the time cost.
  3. Run cargo run --release --example chat to chat with the model.
  4. To specify the location of your safetensors model, use cargo run --release --example chat -- --model /path/to/model.

Or you can download the pre-compiled binaries from the release page and run bash $ chat --model /path/to/model

Use in Your Project

To use in your own rust project, simply add web-rwkv = "0.2" as a dependency in your Cargo.toml. Check examples on how to create the environment, the tokenizer and how to run the model.

Convert Models

You can now download the coverted models here.

You may download the official RWKV World series models from HuggingFace, and convert them via the provided convert_safetensors.py.

An already-converted 0.4B model can be found under assets/models.

Credits