ids_service

Library that allows generating unique ids. It tried to keep cache filled using background threads to fill it. If the cache is empty (the number of requests is too high) an id is calculated on the fly. This will slow down obtaining the id but there is no error when the cache is empty. To generate unique ids the library uses an array of bytes with values from a random number generator and a timestamp as nanoseconds since January 1, 1970 UTC and finally create an hash from both data's.

The library is composed of two modules with similar API but different hash algorithms:

crypto_hash:

The hash algorithm can be one from sha3.

rust_hash:

The hash algorithm is the std::collections::hash_map::DefaultHasher from Rust sdt.

According to your requirements, you may choose which modules you need. The negative impact of using crypto_hash is the performance and resources to run service. Throughput of ids/sec is about half compared to rust DefaultHasher. * The indicative throughput with an Intel i7 using sha512 is above 1'100'000 ids/sec * The throughput is above 1'500'000 ids/sec using Rust Hash.

The size of random block is equal to 2 * the size of hash sha512 = 128 bytes. The size of data block to calculate an unique id using sha512 equal 128 bytes random + 8 bytes of timestamp.

The id can be encode as: * String encoded hexadecimal lower case * String as Base64 * String as Base32 * Json

Quick Start

Use of default cryptographic hash:

```rust extern crate ids_service;

use crate::idsservice::cryptohash::{Encode,IdsService};

fn main() {

/*
 * Create an ids service with:
 * Cache size = 100_000
 * hash algo. = sha256
 * A pool of threads = to the cpu numbers
 */
let mut ids = IdsService::default();
ids.start();
// Optional: Wait cache is filled at 10%
ids.filled_at_percent_event(10).recv().is_ok();
println!("Get an id: {}", ids.get_id().as_hex());
// Graceful Shutdown and Cleanup
ids.stop();

} ```

Use of rust hasher

```rust extern crate ids_service;

use crate::idsservice::rusthash::; use crate::ids_service::common::;

fn main() {

/*
 * Create an ids service with:
 * Cache size = 100'000
 * hash algo. = rust SipHasher
 * A pool of threads = to the cpu numbers
 */
let mut ids = IdsService::default();
ids.start();
// Optional: Wait cache is filled at 100%
ids.filled_event().recv().is_ok();
println!("Get an id: {}", ids.get_id());
// Graceful Shutdown and Cleanup
ids.stop();

} ```

Run example

Examples ended with _rh are using rust hasher

bash cargo run --example examples; cargo run --example examples_rh; bash cargo run --example iterator_example; cargo run --example iterator_example_rh; bash cargo run --example quickstart; cargo run --example quickstart_rh; bash cargo run --example quickstart2; cargo run --example quickstart2_rh; Create ten millions of ids (sha512) and read it all. Throughput with i7, linux, 8 CPU's > 1'100'000 ids/s Throughput with Raspberry PI3 B+ 4 core > 130'000 ids/s bash cargo run --release --example ten_million; cargo run --release --example ten_million_rh;

Platforms

The lib is compiled and tested: 1. x86_64 GNU/Linux (dev + tests) 2. Raspberry PI3 B+ (tests) 3. Windows 10 (tests)

Links

• Rust package library
• git repo master

Releases

• 1.3.0 Changes:
  • Change dependencies rust-crypto to sha3.
    • Note it improve the throughput to around 25% faster
  • Update dependencies
  • Change back to use thread_rng() instead of using one random generator per thread.
• 1.2.1 Changes:
  • Update dependencies
  • Add function
    • pub fn filledatpercentevent(&mut self, percentage : u8) -> mpsc::Receiver_
  • Change the default number of thread from constant to the number of CPU's
  • Solve issue #3
• 1.2.0 Changes:
  • Refactor and split libs.rs to mod.rs
  • Add module rust_hash
  • Change Random number generators from one instance to one instance per thread
  • Minors improvement
• 1.1.0 Changes:
  • Solve issues 1 & 2 with Raspberry and Windows 10 by changing multi threads management
  • Add functions:
    • fn stop(&mut self)
    • fn getidfromcache(&mut self) -> Option\_
    • fn filledevent(&mut self) -> mpsc::Receiver\
    • fn asjson(&self) -> String_ to Trait Encode
    • Solve all clippy warnings
    • Tests with Raspberry Pi3
    • Tests with Windows 10
    • Change default to sha256 and 100_000 cache size
• 1.0.2 Changes:
  • random block size from hash size to 2 * hash size see Birthday problem
  • Change default from sha512 to sha256
• 1.0.1 Add ten_million example
• 1.0.0 Refactor to no generic service, it uses cryptographic hash algorithm only. The internal multi-threads model is more efficient. The API is not compatible with 0.1.6.
• 0.1.6 add function to read configuration from toml file readconfigurationfromtoml_
• 0.1.5 Update readme and comments
• 0.1.4 Add two helper function to create on fly id createidasu128_ and createidasstring_
• 0.1.0 to 0.1.3 Non functional, some adaptation for crates.io and bitbucket pipelines