Deserr is a crate for deserializing data, with the ability to return custom, type-specific errors upon failure. It was also designed with user-facing APIs in mind and thus provides better defaults than serde for this use case.
Unlike serde, deserr does not parse the data in its serialization format itself but offloads the work to other crates. Instead, it deserializes the already-parsed serialized data into the final type. For example:
rust,ignore
// bytes of the serialized value
let s: &str = ".." ;
// parse serialized data using another crate, such as `serde_json`
let json: serde_json::Value = serde_json::from_str(s).unwrap();
// finally deserialize with deserr
let data = T::deserialize_from_value(json.into_value()).unwrap();
// `T` must implement `Deserr`.
The main place where you should use deserr is on your user-facing API, especially if it's supposed to be read by a human. Since deserr gives you full control over your error types, you can improve the quality of your error messages. Here is a little preview of what you can do with deserr:
Let's say I sent this payload to update my Meilisearch settings:
json
{
"filterableAttributes": ["doggo.age", "catto.age"],
"sortableAttributes": ["uploaded_at"],
"typoTolerance": {
"minWordSizeForTypos": {
"oneTypo": 1000, "twoTypo": 80
},
"enabled": true
},
"displayedAttributes": ["*"],
"searchableAttributes": ["doggo.name", "catto.name"]
}
With serde, we don't have much customization; this is the typical kind of message we would get in return:
json
{
"message": "Json deserialize error: invalid value: integer `1000`, expected u8 at line 6 column 21",
"code": "bad_request",
"type": "invalid_request",
"link": "https://docs.meilisearch.com/errors#bad_request"
}
Json deserialize error: invalid value: integer
1000, expected u8 at line 6 column 21
u8, which definitely won't help a user who doesn't know rust or is unfamiliar with types.Since serde returned this error, we cannot know what happened or on which field it happened. Thus, the best we
can do is generate a code bad_request that is common for our whole API. We then use this code to generate
a link to our documentation to help our users. But such a generic link does not help our users because it
can be thrown by every single route of Meilisearch.
json
{
"message": "Invalid value at `.typoTolerance.minWordSizeForTypos.oneTypo`: value: `1000` is too large to be deserialized, maximum value authorized is `255`",
"code": "invalid_settings_typo_tolerance",
"type": "invalid_request",
"link": "https://docs.meilisearch.com/errors#invalid-settings-typo-tolerance"
}
Invalid value at
.typoTolerance.minWordSizeForTypos.oneTypo: value:1000is too large to be deserialized, maximum value authorized is255
.typoTolerance.minWordSizeForTypos.oneTypo. It gives us the faulty field.255.Since deserr called one of our functions in the process, we were able to use a custom error code + link to redirect our user to the documentation specific to this feature and this field.
In Meilisearch, there is another constraint on this minWordSizeForTypos, the twoTypo field must be greater than
the oneType field.
Serde doesn't provide any feature to do that. You could write your own implementation of Deserialize for the
entire sub-object minWordSizeForTypos, but that's generally hard and wouldn't even let you customize the
error type.
Thus, that's the kind of thing you're going to check by hand in your code later on. This is error-prone and
may bring inconsistencies between most of the deserialization error messages and your error message.
With deserr, we provide attributes that allow you to validate your structure once it's deserialized.
It's possible to provide your own function when a field is missing.
rust,ignore
pub fn missing_field<E: DeserializeError>(field: &str, location: ValuePointerRef) -> E {
todo!()
}
At Meilisearch, we use this function to specify a custom error code, but we keep the default error message which is pretty accurate.
It's possible to provide your own function when a field is missing.
rust,ignore
fn unknown_field<E: DeserializeError>(
field: &str,
accepted: &[&str],
location: ValuePointerRef,
) -> E {
todo!()
}
Here is a few ideas we have or would like to implement at Meilisearch;
- In the case of a resource you can PUT with some fields, but can't PATCH all its fields. We can throw a special immutable field x error instead of an unknown field x.
- Detecting when you use the field name of an alternative; for example, we use q to make a query while some Meilisearch alternatives use query.
We could help our users with a did you mean? message that corrects the field to its proper name in Meilisearch.
- Trying to guess what the user was trying to say by computing the levenstein distance
between what the user typed and what is accepted to provide a did you mean? message that attempts to correct typos.
Deserr lets you accumulate multiple errors with its MergeWithError trait while trying to deserialize the value into your type.
This is a good way to improve your user experience by reducing the number of interactions
a user needs to have to fix an invalid payload.
The main parts of deserr are:
1. Deserr<E> is the main trait for deserialization, unlike Serde, it's very easy to deserialize this trait manually, see the implements_deserr_manually.rs file in our examples directory.
2. IntoValue and Value describes the shape that the parsed serialized data must have
3. DeserializeError is the trait that all deserialization errors must conform to
4. MergeWithError<E> describe how to combine multiple errors together. It allows deserr
to return multiple deserialization errors at once.
5. ValuePointerRef and ValuePointer point to locations within the value. They are
used to locate the origin of an error.
6. deserialize<Ret, Val, E> is the main function to use to deserialize a value.
- Ret is the returned value or the structure you want to deserialize.
- Val is the value type you want to deserialize from. Currently, only an implementation for serde_json::Value is provided
in this crate, but you could add your own! Feel free to look into our serde_json module.
- E is the error type that should be used if an error happens during the deserialization.
7. The Deserr derive proc macro
In the following example, we're going to deserialize a structure containing a bunch of fields and uses a custom error type that accumulates all the errors encountered while deserializing the structure.
```rust use deserr::{deserialize, DeserializeError, Deserr, ErrorKind, errors::JsonError, Value, ValueKind, IntoValue, takecfcontent, MergeWithError, ValuePointerRef, ValuePointer}; use serde_json::json; use std::str::FromStr; use std::ops::ControlFlow; use std::fmt; use std::convert::Infallible;
/// This is our custom error type. It'll accumulate multiple JsonError.
struct MyError(Vec
impl DeserializeError for MyError {
/// Create a new error with the custom message.
///
/// Return ControlFlow::Continue to continue deserializing even though an error was encountered.
/// We could return ControlFlow::Break as well to stop right here.
fn errortake_cf_content return the inner error in a ControlFlow<E, E>.
let error = take
let errors = if let Some(MyError(mut errors)) = self_ {
errors.push(error);
errors
} else {
vec![error]
};
ControlFlow::Continue(MyError(errors))
}
}
/// We have to implements MergeWithError between our error type aaand our error type.
impl MergeWithError
struct Search {
#[deserr(default = String::new())]
query: String,
#[deserr(tryfrom(&String) = FromStr::fromstr -> IndexUidError)]
index: IndexUid,
#[deserr(from(String) = From::from)]
field: Wildcard,
#[deserr(default)]
filter: OptionOption it IS mandatory.
limit: Option
/// An IndexUid can only be composed of ascii characters.
struct IndexUid(String); /// If we encounter a non-ascii character this is the error type we're going to throw. struct IndexUidError(char);
impl FromStr for IndexUid { type Err = IndexUidError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
if let Some(c) = s.chars().find(|c| !c.is_ascii()) {
Err(IndexUidError(c))
} else {
Ok(Self(s.to_string()))
}
}
}
impl fmt::Display for IndexUidError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"Encountered invalid character: {}, only ascii characters are accepted in the index",
self.0
)
}
}
/// We need to define how the IndexUidError error is going to be merged with our
/// custom error type.
impl MergeWithErrorJsonError
// type and simply define ourself as an Unexpected error.
let error = takecfcontent(JsonError::error::
/// A Wildcard can either contains a normal value or be a unit wildcard.
enum Wildcard { Wildcard, Value(String), }
impl From
// Here is an example of a typical payload we could deserialize:
let data = deserialize::
// And here is what happens when everything goes wrong at the same time:
let error = deserialize::.query: expected a string, but found a positive integer: 12
Invalid value type at .offset: expected a positive integer, but found a string: \"๐ข\"
Invalid value at .index: Encountered invalid character: ๐ฏ, only ascii characters are accepted in the index
Invalid value type at .field: expected a string, but found a boolean: true
Missing field limit\
");
```
rename_allRename all the fields of the struct according to the given case convention.
The possible values are lowercase, camelCase.
If you need more case conventions, please open an issue; adding more is trivial.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
struct Search {
query: String,
attributestoretrieve: Vec
let data = deserialize::
deny_unknown_fieldsThrow an error when encountering unknown fields. When this attribute is absent, unknown fields are ignored by default.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
struct Search { query: String, }
let err = deserialize::
asserteq!(err.tostring(), "Unknown field doggo: expected one of query");
```
It is also possible to provide a custom function to handle the error.
```rust use deserr::{Deserr, deserialize, ErrorKind, DeserializeError, ValuePointerRef, takecfcontent, errors::JsonError}; use std::convert::Infallible; use serde_json::json;
struct Search { query: String, }
fn unknownfieldssearchE::error returns a ControlFlow<E, E>, which returns the error and indicates
// whether we should keep accumulating errors or not. However, here we simply
// want to retrieve the error's value. This is what take_cf_content does.
match field {
"doggo" => takecfcontent(E::error::
let err = deserialize::
asserteq!(err.tostring(), "Invalid value: can I pet the doggo? uwu");
let err = deserialize::
asserteq!(err.tostring(), "Unknown field catto: expected one of query");
```
tagExternally tag an enum. Deserr does not support internally tagging your enum yet, which means you'll always need to use this attribute if you're deserializing an enum. For complete unit enums, deserr can deserialize their value from a string, though.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
struct Search { query: Query, }
enum Query {
Single {
search: String,
},
Multi {
searches: Vec
let data = deserialize::
fromDeserializing a type from a function instead of a Value.
You need to provide the following information;
1. The input type of the function (here &String)
2. The path of the function (here, we're simply using the std FromStr implementation)
deserr will first try to deserialize the given type using its Deserr<E> implementation.
That means the input type of the from can be complex. Then deserr will call your
function.
See also try_from if your function can fail.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
enum Wildcard { Wildcard, Value(String), }
impl From
let data = deserialize::
let data = deserialize::
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
enum Wildcard { Wildcard, Value(String), }
impl From
struct Search { query: String, #[deserr(from(String) = From::from)] field: Wildcard, }
let data = deserialize::
let data = deserialize::
try_fromTry deserializing a type from a function instead of a Value.
You need to provide the following information;
1. The input type of the function (here &String)
2. The path of the function (here, we're simply using the std FromStr implementation)
3. The error type that this function can return (here Infallible)
deserr will first try to deserialize the given type using its Deserr<E> implementation.
That means the input type of the try_from can be complex. Then deserr will call your
function and accumulate the specified error against the error type of the caller.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json; use std::str::FromStr; use std::fmt;
// Notice how the try_from allows us to leverage the deserr limitation on tuple struct.
struct AsciiString(String);
struct AsciiStringError(char);
impl fmt::Display for AsciiStringError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"Encountered invalid character: {}, only ascii characters are accepted",
self.0
)
}
}
impl std::error::Error for AsciiStringError {}
impl FromStr for AsciiString { type Err = AsciiStringError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
if let Some(c) = s.chars().find(|c| !c.is_ascii()) {
Err(AsciiStringError(c))
} else {
Ok(Self(s.to_string()))
}
}
}
let data = deserialize::
let error = deserialize::๐, only ascii characters are accepted");
```
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json; use std::convert::Infallible; use std::str::FromStr; use std::num::ParseIntError;
struct Search { query: String, #[deserr(tryfrom(&String) = FromStr::fromstr -> ParseIntError)] limit: usize,
}
let data = deserialize::
let error = deserialize::.limit: expected a string, but found a positive integer: 12");
```
validateValidate a structure after it has been deserialized. This is typically useful when your validation logic needs to take multiple fields into account.
```rust use deserr::{Deserr, DeserializeError, ErrorKind, ValuePointerRef, deserialize, errors::JsonError}; use serde_json::json; use std::convert::Infallible;
// __Deserr_E represents the Error returned by the generated Deserr implementation.
struct Range { min: u8, max: u8, }
fn validaterangemax ({}) should be greater than min ({})",
range.max, range.min
),
},
location,
)))
} else {
Ok(range)
}
}
let data = deserialize::
let error = deserialize::max (2) should be greater than min (4)");
```
defaultAllows you to specify a default value for a field.
Note that, unlike serde, by default, Option doesn't automatically use this attribute.
Here you need to explicitly define whether your type can get a default value.
This makes it less error-prone and easier to make an optional field mandatory.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
struct Search {
#[deserr(default)]
query: Option
let data = deserialize::
let data = deserialize::
skipAllows you to skip the deserialization of a field.
It won't show up in the list of fields generated by deny_unknown_fields or in the
UnknownKey variant of the ErrorKind type.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
struct Search {
query: String,
// A field can be skipped if it implements Default or if the default attribute is specified.
#[deserr(skip)]
hidden: usize,
}
let data = deserialize::
// if you try to specify the field, it is ignored
let data = deserialize::
// Here, we're going to see how skip interacts with deny_unknown_fields
struct Search2 {
query: String,
// A field can be skipped if it implements Default.
#[deserr(skip)]
hidden: usize,
}
let error = deserialize::hidden isn't in the list of expected fields + hidden is effectively considered as a non-existing field.
asserteq!(error.to_string(), "Unknown field hidden: expected one of query");
```
mapMap a field after it has been deserialized.
```rust use deserr::{Deserr, deserialize, errors::JsonError}; use serde_json::json;
struct Search { query: String, #[deserr(map = add_one)] limit: usize, }
fn addone(n: usize) -> usize { n.saturatingadd(1) }
let data = deserialize::
// Let's see how map interacts with the default attributes.
struct Search2 { query: String, #[deserr(default, map = add_one)] limit: usize, }
let data = deserialize::map attribute is applied AFTER the default.
assert_eq!(data, Search2 { query: String::from("doggo"), limit: 1 });
```
missing_field_errorGives you the opportunity to customize the error message if this specific field is missing.
```rust use deserr::{Deserr, DeserializeError, ValuePointerRef, ErrorKind, deserialize, errors::JsonError}; use serde_json::json; use std::convert::Infallible;
struct Search { #[deserr(missingfielderror = missingqueryfield)] query: String, limit: usize, }
fn missingqueryfield
let error = deserialize::
errorCustomize the error type that can be returned when deserializing this structure instead of keeping it generic.
```rust use deserr::{Deserr, DeserializeError, ValuePointerRef, ErrorKind, deserialize, errors::JsonError}; use serde_json::json;
struct Search { query: String, limit: usize, }
// As we can see, rust is able to infer the error type.
let data = deserialize::
It can also be used as a field attribute;
```rust use deserr::{Deserr, DeserializeError, ValuePointerRef, ErrorKind, deserialize, errors::JsonError}; use serde_json::json;
// Since the error returned by the Search structure needs to implements MergeWithError<JsonError>
// we also need to specify the error attribute as a JsonError. But as you will see later there are
// other solutions.
struct Search { #[deserr(error = JsonError)] query: A, limit: usize, } ```
where_predicateLet you add where clauses to the Deserr implementation that deserr will generate.
```rust use deserr::{Deserr, DeserializeError, MergeWithError, deserialize, errors::JsonError}; use serde_json::json;
// Here we can constraint the generic __Deserr_E type used by deserr to implements MergeWithError.
// Now instead of constraining the final error type it stays generic if it's able to accumulate with
// with a JsonError.
struct Search { #[deserr(error = JsonError)] query: A, limit: usize, } ```
needs_predicateAutomatically adds where_predicate = FieldType: Deserr<ErrType> for each field with this attribute.
```rust use deserr::{Deserr, DeserializeError, MergeWithError, deserialize, errors::JsonError}; use serde_json::json;
struct Search { #[deserr(needs_predicate)] query: A, limit: usize, } ```
Is strictly equivalent to the following:
```rust use deserr::{Deserr, DeserializeError, MergeWithError, deserialize, errors::JsonError}; use serde_json::json;
// __Deserr_E represents the Error returned by the generated Deserr implementation.
struct Search { query: A, limit: usize, } ```
Since deserr needs to first deserialize the payload into a generic Value that allocates
a lot of memory before creating your structure, it's a lot slower than serde.
For example, at Meilisearch for our search route, in case of a valid payload, we observed a 400% slowdown (4 times slower). That made our search request deserialize in 2ยตs instead of 500ns. This is fast enough for most use cases but could be an issue if most of your time is spent deserializing.
| datastructure | serde | deserr | note | |---------------------|-------|--------|------| | Struct | yes | yes | | | Tuple struct | yes | no | | | Untagged Enum | yes | no | | | Untagged unit Enum | yes | yes | | | Tagged Enum | yes | yes | |
| features | serde | deserr | note |
|---------------------|-------|--------|------|
| rename | yes | no | |
| renameall | yes | yes | |
| denyunknownfields | yes | yes | With deserr you can call a custom function when an unknown field is encountered |
| tag | yes | yes | |
| tag+content | yes | no | |
| untagged | yes | no | it's only supported for unit enums |
| bound | yes | no | Can be emulated with where_predicate |
| default | yes | no | |
| remote | yes | no | |
| transparent | yes | no | |
| from | yes | yes | |
| tryfrom | yes | yes | |
| into | yes | no | |
| crate | yes | no | |
| validate | no | yes | Allows you to validate the content of struct after it has been deserialized |
| error | no | yes | Specify the error type that should be used while deserializing this structure |
| where_predicate | no | yes | Let you add where clauses to the generated Deserr implementation |
| features | serde | deserr | note |
|---------------------|-------|--------|------|
| rename | yes | no | |
| alias | yes | no | |
| default | yes | yes | |
| flatten | yes | no | serde doesn't support flattening + denying unknown field |
| skip | yes | yes | |
| deserializewith | yes | no | But it's kinda emulated with from and try_from |
| with | yes | no | |
| borrow | yes | no | deserr does not support types with references |
| bound | yes | no | |
| map | no | yes | Allows you to map the value after it was deserialized |
| from | no | yes | Deserialize this field from an infallible function |
| tryfrom | no | yes | Deserialize this field from a fallible function |
| missingfielderror | no | yes | Allows you to return a custom error if this field is missing |
| error | no | yes | Specify the error type that should be used while deserializing this field |
serde-jsonImport serde_json and provide;
- An implementation of deserr::IntoValue for serde_json::Value which make it easy to use both crate together.
- A default implementation of the JsonError type that provide the best generic error messages possible.
serde-csImport serde-cs and provide;
- An implementation of Deserr for serde_cs::CS<R>.
actix-webImport actix-web and futures and provide;
- An implementation of a json actix-web extractor if used with the serde-json feature.
- An implementation of ResponseError for the JsonError type if used with the serde-json feature.
At Meilisearch, we wanted to customize the error code we return when we fail the deserialization of a specific field. Some error messages were also not clear at all and impossible to edit.
At Meilisearch we're already using deserr in production; thus, it's well maintained.
Currently, you can read our examples in the examples directory of this repository.
You can also look at our integration test; each attribute has a simple-to-read test.
And obviously, you can read the code of Meilisearch where deserr is used on all our routes.
Please, if you think there is a bug in this lib or would like a new feature, open an issue or a discussion. If you would like to chat more directly with us, you can join us on discord at https://discord.com/invite/meilisearch
Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.