pqrs is a tool which deserializes compiled protobuf messages given a set of pre-compiled .fdset files.
pqrs is on crates.io: cargo install pq. You can also download a static binary from the releases page.
Read the manpage!
To set up, put your *.fdset files in ~/.pq:
$ protoc -o dog.fdset dog.proto
$ protoc -o person.fdset person.proto
$ cp *.fdset ~/.pq/
Pipe a single compiled protobuf message to pq:
$ ./tests/python/testbench.py single | pq | jq
{
"age": 4,
"breed": "poodle",
"temperament": "excited"
}
$ ./tests/python/testbench.py single | pq | jq
{
"id": 2,
"name": "raffi"
}
Pipe a dirty (extra leading/trailing chars) to pq:
$ ./tests/python/testbench.py dirty | pq | jq
{
"id": 1,
"name": "vahaken"
}
Pipe a varint32-delimited stream to pq:
$ ./tests/python/testbench.py stream | pq --stream="varint" | jq
{
"age": 10,
"breed": "gsd",
"temperament": "aggressive"
}
{
"id": 3,
"name": "khosrov"
}
{
"age": 8,
"breed": "rottweiler",
"temperament": "aggressive"
}
{
"id": 2,
"name": "vahaken"
}
Sometimes protobuf is sent in ways that add their own delimiters. E.g. if you send varint-delimited Protobuf over kafka, to use pqrs, you would need to deal with the message delimiters inserted by kafka-console-consumer.sh or whatever your command-line kafka consumer is that you are piping to pq.
Tentative support for this is added in the form of a --trail="\n\n" command-line option.