Moka

GitHub Actions crates.io release docs dependency status coverage status license FOSSA Status

Moka is a fast, concurrent cache library for Rust. Moka is inspired by the Caffeine library for Java.

Moka provides cache implementations on top of hash maps. They support full concurrency of retrievals and a high expected concurrency for updates. Moka also provides a non-thread-safe cache implementation for single thread applications.

All caches perform a best-effort bounding of a hash map using an entry replacement algorithm to determine which entries to evict when the capacity is exceeded.

Features

Moka in Production

Moka is powering production services as well as embedded Linux devices like home routers. Here are some highlights:

Change Log

Usage

Add this to your Cargo.toml:

toml [dependencies] moka = "0.8"

To use the asynchronous cache, enable a crate feature called "future".

toml [dependencies] moka = { version = "0.8", features = ["future"] }

Example: Synchronous Cache

The thread-safe, synchronous caches are defined in the sync module.

Cache entries are manually added using insert or get_with method, and are stored in the cache until either evicted or manually invalidated.

Here's an example of reading and updating a cache by using multiple threads:

```rust // Use the synchronous cache. use moka::sync::Cache;

use std::thread;

fn value(n: usize) -> String { format!("value {}", n) }

fn main() { const NUMTHREADS: usize = 16; const NUMKEYSPERTHREAD: usize = 64;

// Create a cache that can store up to 10,000 entries.
let cache = Cache::new(10_000);

// Spawn threads and read and update the cache simultaneously.
let threads: Vec<_> = (0..NUM_THREADS)
    .map(|i| {
        // To share the same cache across the threads, clone it.
        // This is a cheap operation.
        let my_cache = cache.clone();
        let start = i * NUM_KEYS_PER_THREAD;
        let end = (i + 1) * NUM_KEYS_PER_THREAD;

        thread::spawn(move || {
            // Insert 64 entries. (NUM_KEYS_PER_THREAD = 64)
            for key in start..end {
                my_cache.insert(key, value(key));
                // get() returns Option<String>, a clone of the stored value.
                assert_eq!(my_cache.get(&key), Some(value(key)));
            }

            // Invalidate every 4 element of the inserted entries.
            for key in (start..end).step_by(4) {
                my_cache.invalidate(&key);
            }
        })
    })
    .collect();

// Wait for all threads to complete.
threads.into_iter().for_each(|t| t.join().expect("Failed"));

// Verify the result.
for key in 0..(NUM_THREADS * NUM_KEYS_PER_THREAD) {
    if key % 4 == 0 {
        assert_eq!(cache.get(&key), None);
    } else {
        assert_eq!(cache.get(&key), Some(value(key)));
    }
}

} ```

If you want to atomically initialize and insert a value when the key is not present, you might want to check the document for other insertion methods get_with and try_get_with.

Example: Asynchronous Cache

The asynchronous (futures aware) cache is defined in the future module. It works with asynchronous runtime such as Tokio, async-std or actix-rt. To use the asynchronous cache, enable a crate feature called "future".

Cache entries are manually added using an insert method, and are stored in the cache until either evicted or manually invalidated:

Here is a similar program to the previous example, but using asynchronous cache with Tokio runtime:

```rust,ignore // Cargo.toml // // [dependencies] // moka = { version = "0.8", features = ["future"] } // tokio = { version = "1", features = ["rt-multi-thread", "macros" ] } // futures-util = "0.3"

// Use the asynchronous cache. use moka::future::Cache;

[tokio::main]

async fn main() { const NUMTASKS: usize = 16; const NUMKEYSPERTASK: usize = 64;

fn value(n: usize) -> String {
    format!("value {}", n)
}

// Create a cache that can store up to 10,000 entries.
let cache = Cache::new(10_000);

// Spawn async tasks and write to and read from the cache.
let tasks: Vec<_> = (0..NUM_TASKS)
    .map(|i| {
        // To share the same cache across the async tasks, clone it.
        // This is a cheap operation.
        let my_cache = cache.clone();
        let start = i * NUM_KEYS_PER_TASK;
        let end = (i + 1) * NUM_KEYS_PER_TASK;

        tokio::spawn(async move {
            // Insert 64 entries. (NUM_KEYS_PER_TASK = 64)
            for key in start..end {
                // insert() is an async method, so await it.
                my_cache.insert(key, value(key)).await;
                // get() returns Option<String>, a clone of the stored value.
                assert_eq!(my_cache.get(&key), Some(value(key)));
            }

            // Invalidate every 4 element of the inserted entries.
            for key in (start..end).step_by(4) {
                // invalidate() is an async method, so await it.
                my_cache.invalidate(&key).await;
            }
        })
    })
    .collect();

// Wait for all tasks to complete.
futures_util::future::join_all(tasks).await;

// Verify the result.
for key in 0..(NUM_TASKS * NUM_KEYS_PER_TASK) {
    if key % 4 == 0 {
        assert_eq!(cache.get(&key), None);
    } else {
        assert_eq!(cache.get(&key), Some(value(key)));
    }
}

} ```

If you want to atomically initialize and insert a value when the key is not present, you might want to check the document for other insertion methods get_with and try_get_with.

Avoiding to clone the value at get

For the concurrent caches (sync and future caches), the return type of get method is Option<V> instead of Option<&V>, where V is the value type. Every time get is called for an existing key, it creates a clone of the stored value V and returns it. This is because the Cache allows concurrent updates from threads so a value stored in the cache can be dropped or replaced at any time by any other thread. get cannot return a reference &V as it is impossible to guarantee the value outlives the reference.

If you want to store values that will be expensive to clone, wrap them by std::sync::Arc before storing in a cache. Arc is a thread-safe reference-counted pointer and its clone() method is cheap.

```rust,ignore use std::sync::Arc;

let key = ... let large_value = vec![0u8; 2 * 1024 * 1024]; // 2 MiB

// When insert, wrap the largevalue by Arc. cache.insert(key.clone(), Arc::new(largevalue));

// get() will call Arc::clone() on the stored value, which is cheap. cache.get(&key); ```

Example: Size Aware Eviction

If different cache entries have different "weights" — e.g. each entry has different memory footprints — you can specify a weigher closure at the cache creation time. The closure should return a weighted size (relative size) of an entry in u32, and the cache will evict entries when the total weighted size exceeds its max_capacity.

```rust use std::convert::TryInto; use moka::sync::Cache;

fn main() { let cache = Cache::builder() // A weigher closure takes &K and &V and returns a u32 representing the // relative size of the entry. Here, we use the byte length of the value // String as the size. .weigher(|key, value: &String| -> u32 { value.len().tryinto().unwrapor(u32::MAX) }) // This cache will hold up to 32MiB of values. .maxcapacity(32 * 1024 * 1024) .build(); cache.insert(0, "zero".to_string()); } ```

Note that weighted sizes are not used when making eviction selections.

Example: Expiration Policies

Moka supports the following expiration policies:

To set them, use the CacheBuilder.

```rust use moka::sync::Cache; use std::time::Duration;

fn main() { let cache = Cache::builder() // Time to live (TTL): 30 minutes .timetolive(Duration::fromsecs(30 * 60)) // Time to idle (TTI): 5 minutes .timetoidle(Duration::fromsecs( 5 * 60)) // Create the cache. .build();

// This entry will expire after 5 minutes (TTI) if there is no get().
cache.insert(0, "zero");

// This get() will extend the entry life for another 5 minutes.
cache.get(&0);

// Even though we keep calling get(), the entry will expire
// after 30 minutes (TTL) from the insert().

} ```

A note on expiration policies

The cache builders will panic if configured with either time_to_live or time to idle longer than 1000 years. This is done to protect against overflow when computing key expiration.

Hashing Algorithm

By default, a cache uses a hashing algorithm selected to provide resistance against HashDoS attacks.

The default hashing algorithm is the one used by std::collections::HashMap, which is currently SipHash 1-3, though this is subject to change at any point in the future.

While its performance is very competitive for medium sized keys, other hashing algorithms will outperform it for small keys such as integers as well as large keys such as long strings. However those algorithms will typically not protect against attacks such as HashDoS.

The hashing algorithm can be replaced on a per-Cache basis using the build_with_hasher method of the CacheBuilder. Many alternative algorithms are available on crates.io, such as the aHash crate.

Minimum Supported Rust Versions

This crate's minimum supported Rust versions (MSRV) are the followings:

| Feature | Enabled by default? | MSRV | |:-----------|:-------------------:|:-----------:| | no feature | | Rust 1.51.0 | | atomic64 | yes | Rust 1.51.0 | | future | | Rust 1.51.0 | | dash | | Rust 1.51.0 |

If only the default features are enabled, MSRV will be updated conservatively. When using other features, like future, MSRV might be updated more frequently, up to the latest stable. In both cases, increasing MSRV is not considered a semver-breaking change.

Resolving Compile Errors on Some 32-bit Platforms

On some 32-bit target platforms including the followings, you may encounter compile errors:

console error[E0432]: unresolved import `std::sync::atomic::AtomicU64` --> ... /moka-0.5.3/src/sync.rs:10:30 | 10 | atomic::{AtomicBool, AtomicU64, Ordering}, | ^^^^^^^^^ | | | no `AtomicU64` in `sync::atomic`

Such errors can occur because std::sync::atomic::AtomicU64 is not provided on these platforms but Moka uses it.

You can resolve the errors by disabling atomic64 feature, which is one of the default features of Moka. Edit your Cargo.toml to add default-features = false to the dependency declaration.

```toml:Cargo.toml [dependencies] moka = { version = "0.8", default-feautures = false }

Or

moka = { version = "0.8", default-feautures = false, features = ["future"] } ```

This will make Moka to switch to a fall-back implementation, so it will compile.

Developing Moka

Running All Tests

To run all tests including future feature and doc tests on the README, use the following command:

console $ RUSTFLAGS='--cfg skeptic --cfg trybuild' cargo test --all-features

Running All Tests without Default Features

console $ RUSTFLAGS='--cfg skeptic --cfg trybuild' cargo test \ --no-default-features --features future

Generating the Doc

console $ cargo +nightly -Z unstable-options --config 'build.rustdocflags="--cfg docsrs"' \ doc --no-deps --features 'future, dash'

Road Map

About the Name

Moka is named after the moka pot, a stove-top coffee maker that brews espresso-like coffee using boiling water pressurized by steam.

This name would imply the following facts and hopes:

Credits

cht

The source files of the concurrent hash table under moka::cht module were copied from the cht crate v0.4.1 and modified by us. We did so for better integration.

The cht is authored by Gregory Meyer and its v0.4.1 and earlier versions are licensed under the MIT license.

License

Moka is distributed under either of

at your option.

See LICENSE-MIT and LICENSE-APACHE for details.

FOSSA Status