tinysearch

CI

tinysearch is a lightweight, fast, full-text search engine. It is designed for static websites.

tinysearch is written in Rust, and then compiled to WebAssembly to run in a browser.\ It can be used together with static site generators such as Jekyll, Hugo, Zola, Cobalt, or Pelican.

Demo

Is it tiny?

The test index file of my blog with around 40 posts creates a WASM payload of 99kB (49kB gzipped, 40kB brotli).\ That is smaller than the demo image above; so yes.

How it works

tinysearch is a Rust/WASM port of the Python code from the article "Writing a full-text search engine using Bloom filters". It can be seen as an alternative to lunr.js and elasticlunr, which are too heavy for smaller websites and load a lot of JavaScript.

Under the hood it uses a Xor Filter — a datastructure for fast approximation of set membership that is smaller than bloom and cuckoo filters. Each blog post gets converted into a filter that will then be serialized to a binary blob using bincode. Please note that the underlying technologies are subject to change.

Limitations

Installation

wasm-pack is required to build the WASM module. Install it with

sh cargo install wasm-pack

To optimize the JavaScript output, you'll also need terser:

npm install terser -g

If you want to make the WebAssembly as small as possible, we recommend to install binaryen as well. On macOS you can install it with homebrew:

sh brew install binaryen

Alternatively, you can download the binary from the release page or use your OS package manager.

After that, you can install tinysearch itself:

cargo install tinysearch

Usage

A JSON file, which contains the content to index, is required as an input. Please take a look at the example file.

ℹ️ The body field in the JSON document is optional and can be skipped to just index post titles.

Once you created the index, you can run

tinysearch fixtures/index.json

This will create a WASM module and the JavaScript glue code to integrate it into your website. You can open the demo.html from any webserver to see the result.

For example, Python has a built-in webserver that can be used for a quick test:

python3 -m http.server

then browse to http://0.0.0.0:8000/demo.html to run the demo.

You can also take a look at the code examples for different static site generators here.

Advanced Usage

For advanced usage options, run

tinysearch --help

Please check what's required to host WebAssembly in production -- you will need to explicitly set gzip mime types.

Docker

If you don't have a full Rust setup available, you can also use our nightly-built Docker images.

Here is how to quickly try tinysearch with Docker:

```sh

Download a sample blog index from endler.dev

curl -O https://raw.githubusercontent.com/tinysearch/tinysearch/master/fixtures/index.json

Create the WASM output

docker run -v $PWD:/app tinysearch/cli --engine-version path=\"/engine\" --path /app/wasm_output /app/index.json ```

By default, the most recent stable Alpine Rust image is used. To get nightly, run

sh docker build --build-arg RUST_IMAGE=rustlang/rust:nightly-alpine -t tinysearch/cli:nightly .

Advanced Docker Build Args

Github action

To integrate tinysearch in continuous deployment pipelines, a github action is available.

yaml - name: Build tinysearch uses: leonhfr/tinysearch-action@v1 with: index: public/index.json output_dir: public/wasm output_types: | wasm

Users

The following websites use tinysearch:

Are you using tinysearch, too? Add your site here!

Maintainers

License

tinysearch is licensed under either of

at your option.