gguf

A small utility library for parsing GGUF file info. See also GGML library.

Running locally

```bash $ cargo run --features bin -q -- --help A small utility to parse GGUF files

Usage: gguf-info [OPTIONS]

Arguments: The path to the file to read

Options: --read-buffer-size Size of read buffer (grows linearly) [default: 1000000] -t, --output-format [default: table] [possible values: yaml, json, table] -h, --help Print help -V, --version Print version ```

```bash

check with your own gguf file

$ cargo run --features bin -- ~/GitHub/llama/llama-2-7b/ggml-model-Q40.gguf Metadata: +----+----------------------------------------+---------------+-----------------------+ | # | Key | Type | Value | +=====================================================================================+ | 1 | general.architecture | String | llama | |----+----------------------------------------+---------------+-----------------------| | 2 | general.name | String | LLaMA v2 | |----+----------------------------------------+---------------+-----------------------| | 3 | llama.contextlength | Uint32 | 4096 | |----+----------------------------------------+---------------+-----------------------| | 4 | llama.embeddinglength | Uint32 | 4096 | |----+----------------------------------------+---------------+-----------------------| | 5 | llama.blockcount | Uint32 | 32 | |----+----------------------------------------+---------------+-----------------------| | 6 | llama.feedforwardlength | Uint32 | 11008 | |----+----------------------------------------+---------------+-----------------------| | 7 | llama.rope.dimensioncount | Uint32 | 128 | |----+----------------------------------------+---------------+-----------------------| | 8 | llama.attention.headcount | Uint32 | 32 | |----+----------------------------------------+---------------+-----------------------| | 9 | llama.attention.headcountkv | Uint32 | 32 | |----+----------------------------------------+---------------+-----------------------| | 10 | llama.attention.layernormrmsepsilon | Float32 | 0.00001 | |----+----------------------------------------+---------------+-----------------------| | 11 | general.filetype | Uint32 | 2 | |----+----------------------------------------+---------------+-----------------------| | 12 | tokenizer.ggml.model | String | llama | |----+----------------------------------------+---------------+-----------------------| | 13 | tokenizer.ggml.tokens | Array (32000) | , , , ... | |----+----------------------------------------+---------------+-----------------------| | 14 | tokenizer.ggml.scores | Array (32000) | 0, 0, 0, ... | |----+----------------------------------------+---------------+-----------------------| | 15 | tokenizer.ggml.tokentype | Array (32000) | 2, 3, 3, ... | |----+----------------------------------------+---------------+-----------------------| | 16 | general.quantizationversion | Uint32 | 2 | +----+----------------------------------------+---------------+-----------------------+ Tensors: +-----+---------------------------+------+---------------+------------+ | # | Name | Type | Dimensions | Offset | +=====================================================================+ | 1 | tokenembd.weight | Q40 | [4096, 32000] | 0 | |-----+---------------------------+------+---------------+------------| | 2 | output_norm.weight | F32 | [4096] | 73728000 | |-----+---------------------------+------+---------------+------------| | 3 | output.weight | Q6K | [4096, 32000] | 73744384 | |-----+---------------------------+------+---------------+------------| ```