The motivation for this product is two fold:
To be able to map a byte slice &[u8] , typically acquired from a network, into a rust struct datamodel by simply using derive macro annotations and attributes auto-generate necessary code.
This comes very handy when the byte slice is not serialized using one of the existing and widely available protocols. Ex: An application specific C-Struct.
To be the fastest byte stream serializer/deserializer on the market for latency sensitive use cases. Benchmark results below show a performance summary of serializing & deserializing a Reference Struct using different frameworks available:
byteserde - ~15ns read/write bincode - ~15ns read / ~100ns writermp-serde - ~215ns read/writeserde_json - ~600ns read/write - understandably slow due to strings usageuse and when to avoid this frameworkIf you work with network protocols that deliver data in a byte stream format that is not matching one of the widely available standards, ex: bincode, protobuf. Use this product to efficiently map your byte stream into a rust struct.
You have a latency sensitive usecase. Note that this protocol does not add any schema information during serialization process and hence is equivalent of dumping the memory layout of the struct without padding
Example of protocols that are a perfect fit for this framework.
byte stream is serialized or deserialized using a wideley available standard avoid this framework and instead the that respective standard to work with the byte streamcontains derive macros that generates byteserde@crates.io traits
#[derive(ByteSerializeStack)] - generates ByteSerializeStack trait
#[derive(ByteSerializeHeap)] - generates ByteSerializeHeap trait
#[derive(ByteDeserializeSlice)] - generates ByteDeserializeSlice<T> trait
#[derive(ByteSerializedSizeOf)] - generates ByteSerializedSizeOf trait - this trait provides an associated method byte_size() which gives you a struct memory size in bytes without alignment. However it does not support types which heap allocate, ex: Vectors, Strings, or their derivations.
#[derive(ByteSerializedLenOf)] - generates ByteSerializedLenOf trait - this trait provides an instance method byte_len(&self) which gives you memory size in bytes without alignment of specific instance. It exists specifically to deal with types that ByteSerializedSizeOf trait does not support
Highlights
<CAP> - provides ultra fast serializer into a pre allocated byte array [u8; CAP] on stack, hence the name, it is very fast but at the cost of you needing to specify the size of the LARGEST struct you will attempt to serialize. If you reach the boundary of this preallocated byte array, your serialization will fail. This utility provides a reset features, which moves the internal counter to the begining, and allows you to recycle the buffer multiple times. works for structs that implement ByteSerializeStack trait
ByteSerializerHeap - provides a fast enough for most speed by serializing into a byte vector Vec<u8>, hence the name. This utility trades some performance in return for not having to worry about knowing the LARGEST struct size in advance.
works for structs that implement ByteSerializeHeap trait
ByteDeserializerSlice - takes a byte stream &[u8] irrespctive of heap vs stack allocation and turns it into a struct
structs that implement ByteDeserializeSlice<T> traitbyte stream, follow example section for more details.