OptiVorbis
A library and application for lossless, format-preserving, two-pass optimization and repair of Vorbis data, reducing its size without altering any audio information.
OptiVorbis does lossless optimizations and repairs of complete, seekable Vorbis I audio streams, usually contained in Ogg Vorbis (.ogg) files, as defined in the Vorbis I specification. It leverages the great flexibility the Vorbis I stream setup configuration header provides to achieve its goals.
The optimization is lossless: streams processed by this library are guaranteed to be decoded to the same audio samples as before by any sane decoder. The internal file structure may differ substantially, but these differences are transparent for end-users, as the resulting streams still conform to the specification. At the user's discretion, it is possible to save even more space by removing comments (track title, author, etc.) and encoder version information.
In addition, OptiVorbis' understanding of the Vorbis format and container encapsulations, combined with the somewhat more lenient, purpose-built parsers it uses, provide it with some repair capabilities. It also tends to output more detailed and actionable error information on failure than other tools, rendering it suitable for sanity-checking Vorbis streams.
In a nutshell, the optimization works by doing two passes over the Vorbis stream data, which requires the whole stream to be available and seekable, usually either in a memory buffer or a file:
OptiVorbis is officially distributed in three ways:
If you are an end-user looking to optimize files, either the demo web page or the CLI application is all you need. On the other hand, if you are a developer, you should look into the CLI or the Rust library.
Just visit the website: no installation required! Only a modern web browser that supports the required technologies is needed. Notably, Internet Explorer is not supported, but you should no longer be using it.
Download the appropriate CLI executable from GitHub Releases, according to your operating system and CPU architecture. The executables are statically linked, so they are entirely self-contained and do not require any system configuration to run.
If you are using a Unix-like OS (Linux, macOS), remember that the executable must have the execute permission to work. This permission may be granted with the chmod command: chmod +x optivorbis.
If you want to go bleeding edge and try out code that was not yet released, you can get the latest CLI executables from the GitHub Actions CI workflow. No guarantees are made about the stability of these builds.
Like any other dependency, add it to the Cargo.toml manifest of your project, and start calling its API as you like. Check out its page at crates.io and the API docs at docs.rs to get started.
The minimum supported Rust version (MSRV) for every package in this repository is 1.61. Bumping this version is not considered a breaking change for semantic versioning purposes. We will try to do it only when we estimate that such a bump would not cause widespread inconvenience or breakage.
By now, it should be obvious how to use the demo web page or Rust library. The CLI arguments follow a well-defined syntax. Several options are accepted to customize the optimization process and affect the output and operation of the CLI. The most important one to get started is --help, which shows the following usage help:
``` Usage: optivorbis [OPTION]...
Options: -h, --help Prints information about the accepted command line arguments and exits. --version Prints version and copyright information, then exits. -q, --quiet When enabled, the program will only print error messages, unless -h is specified. -v, --verbose Increases the verbosity of the messages. Can be repeated several times. -r, --remuxer REMUXER The remuxer to use for managing the encapsulation of Vorbis streams in a container. If not specified, it will be automatically deduced from the extension of the output file. Available remuxers: ogg2ogg --vendorstringaction VENDOR-STRING-ACTION Changes how the vendor string contained in the Vorbis identification header will be dealt with. Available actions: copy, replace, appendTag, appendShortTag, empty --commentfieldsaction COMMENT-FIELDS-ACTION Changes how the user comment fields contained in the Vorbis comment header will be dealt with. Available actions: copy, delete --remuxeroption OPTION=VALUE Sets a remuxer-specific option to a value. ----------------------- ogg2ogg remuxer options ----------------------- - randomizestreamserials=BOOLEAN If set to true, the stream serials will be randomized, following the intent of the Ogg specification. Set to false to disable this behavior and have more control over the serials. The default value is true. - firststreamserialoffset=INTEGER A zero or positive integer that sets the offset that will be added to the serial of the first stream. When not randomizing stream serials, the offset matches the serial that will be used for the first stream. The default value is 0. - ignorestartsample_offset=BOOLEAN Sets whether a non-zero calculated granule position for the first audio sample will be honored when recomputing granule positions in the generated Ogg file or not. This usually is a good thing, but for increased compatibility with some players or dedicated purposes it may be advised to ignore this offset. The default value is false. ```
OptiVorbis has been developed as a part of a master's thesis and extensively tested before its public release with a dataset of 1783 randomly selected, valid Ogg Vorbis files downloaded from Freesound, totaling 164 hours and 16.58 GiB of sound data. OptiVorbis passed all the defined unit, integration, and system tests, achieving a size reduction for every analyzed file without altering the audio samples as decoded by oggdec (from the Debian vorbis-tools package, version 1.4.2). The only exceptions were four files that had their granule position data corruption fixed by OptiVorbis. The dataset comprised all kinds of Ogg Vorbis files containing mono, stereo, and surround sounds at varying sampling rates, generated by a wide variety of encoders. It was optimized in about half an hour, parallelizing the work across six CPU cores.
The following boxplot represents the distribution of relative per-file size reductions achieved by OptiVorbis.
Even though the optimization potential may vary substantially, every file had its size reduced. The achieved reduction was significantly better than using generic compression tools on the Ogg Vorbis files, such as xz. Interestingly, the file size reduction distribution is slightly skewed towards higher reduction rates, as evidenced by the average (11.35%) being slightly higher than the median (9.62%). Half of the analyzed files had their size reduced between 4.91% and 13.44%.
Notably, the testing techniques included fuzzing, employing the AFL++ fuzzer. With the help of AFL++, the optimization of 600 million mutated Ogg Vorbis files was tested, discovering tens of failing runtime assertions that were fixed.
The OptiVorbis library is licensed under either of
LICENSE or https://www.gnu.org/licenses/agpl-3.0.html)LICENSE.BSD-3-Clause or https://opensource.org/licenses/BSD-3-Clause)at your option. The OptiVorbis CLI is licensed under the GNU Affero General Public License, Version 3 only. Other components (the web demo, etc.) are licensed according to the license files present in their root directory and their package metadata, if any.
Pull requests are accepted. Feel free to contribute if you can improve some aspect of the OptiVorbis project!
Contributions include, but are not limited to:
Code contributions must pass CI checks and be deemed of enough quality by a repository maintainer to be merged. OptiVorbis is structured as a standard Cargo workspace with several packages:
packages/optivorbis: the OptiVorbis Rust library.packages/optivorbis_afl_fuzz_target: an AFL++ target for fuzzing OptiVorbis. See also the related scripts/afl-fuzz.sh script.packages/optivorbis_cli: the OptiVorbis CLI.packages/vorbis_bitpack: an implementation of bitwise reading and writing operations according to the Vorbis bitpacking convention.The website (web) is a standard npm project that uses WebPack and the OptiVorbis library WASM target JS bindings, built with wasm-pack.
We welcome friendly talk about the project, including questions, congratulations, and suggestions. Head to the GitHub Discussions page to interact with fellow users, contributors and developers.
Thanks goes to these wonderful people (emoji key):
 Alejandro GonzΓ‘lez π» π π¨ π€ π§ π π£ π π¬ |
 victorlf4 π€ |
 Miguel π¨ |
 sya-ri π€ |
This project follows the all-contributors specification. Contributions of any kind welcome!
If you found OptiVorbis useful, you may be interested in programs such as:
rogg: a small library for manipulating Ogg Vorbis files, which offers advanced and unusual features such as adjusting granule positions, replacing stream serial numbers, and recomputing the CRC of every Ogg page.oggz: a library and set of command-line tools for low-level inspection and manipulation of Ogg Vorbis files.revorb: a command-line tool for recomputing granule positions of Ogg Vorbis files. OptiVorbis does such recomputation automatically, too.rehuff: a proprietary, buggy, proof of concept program for optimizing Vorbis streams from 2002, written by Segher Boessenkool. OptiVorbis expands upon the optimization techniques said to be implemented in rehuff, offering a much more finished, reliable solution that is open-source.As a concession to implementation simplicity, Vorbis streams that use the floor signal component format of type 0 are not supported. According to the Vorbis I specification, this format is "of limited modern use" and has been effectively deprecated by every known Vorbis encoder for more than 20 years, so streams with this floor type should be extremely rare to find. Pull requests that address this limitation are welcome.
Due to the two-pass optimization algorithm described above, OptiVorbis is not readily applicable for live-streaming use cases.
The Vorbis I setup header codebook format is vulnerable to denial of service attacks, as extremely dense prefix code trees, which take a significantly long time to parse, are valid according to the specification. OptiVorbis does not impose a depth or density limit in such trees, which guarantees its interoperability, but renders it vulnerable to specially-crafted files. This may be addressed in the future as information about the interoperability and mitigation impact of limiting the tree depth is gathered. In the meantime, applications dealing with untrusted files should be aware of this and resort to using OS features to bound resource consumption when applicable.