tinyvector - the tiny, least-dumb, speedy vector embedding database.
🦀 rewrite of 0hq's tinyvector
In most cases, most vector databases are overkill for something simple like: 1. Using embeddings to chat with your documents. Most document search is nowhere close to what you'd need to justify accelerating search speed with HNSW or FAISS. 2. Doing search for your website or store. Unless you're selling 1,000,000 items, you don't need Pinecone. 3. Performing complex search queries on a very large database. Even if you have 2 million embeddings, this might still be the better option due to vector databases struggling with complex filtering. Tinyvector doesn't support metadata/filtering just yet, but it's very easy for you to add that yourself.
What are embeddings?
As simple as possible: Embeddings are a way to compare similar things, in the same way humans compare similar things, by converting text into a small list of numbers. Similar pieces of text will have similar numbers, different ones have very different numbers.
Read OpenAI's explanation.