Cephalon Knowledge Base Assistant

Cephalon is a library to create NLP powered knowledge base assistant with privacy in mind. Cephalon can provide: * Single Source of all documentation. ✅ * Semantic Search ✅ * Multi-Modality [Schduled to start in late Fall 2023] ❔ * Support for Images [Scheduled to start in late Fall 2023] ❔

Creating a Knowledge Base Assistant

You can create a knowledeg base with: cephalon init or cephalon create your-knowledge-base-assistant After that move all the documentation that you might have into your project directory and run cephalon build

You can query the knowledge base by entering a query like this.

``` cephalon answer 'your-query-or-text'

```

Using Cephalon in your code-base

Creating a new cephalon project

``` use cephalon::knowledge_base::{ Cephalon, util };

fn main(){ let currentdirpath:PathBuf = std::env::currentdir().unwrap(); let _cephalonknowledgebase = Cephalon::new(currentdir_path); } ``` This will create a .cephalon directory in the project directory. All, the data related to cephalon will be kept in there.

Scanning files and building Index and Database

use cephalon::knowledge_base::{ Cephalon, util }; fn main(){ let current_dir_path:PathBuf = std::env::current_dir().unwrap(); //Load and existing cephalon project let cephalon_knowledge_base = Cephalon::load(current_dir_path.clone()); //Point to the directory where the files are located. cephalon_knowledge_base.search_and_build_index(&current_dir_path); } This will scan all the files in the given directory. Then if the file type is supported by the program, it will extract text from them, split it into chunks of 256 characters, and save it in the cephalon data base. It will also create embeddings for those files via a Sentence-Embedding model and then upload them to an index and save the index in .cephalon directory. At, the moment the files need to be in the same directory as .cephalon directory. However, in future it will allow you to index any file or directory from any path.

Searching for a specific text

``` use cephalon::knowledgebase::{ Cephalon, util }; fn main(){ let currentdirpath:PathBuf = std::env::currentdir().unwrap(); //Load a cephalon that is already built. let cephalonknowledgebase = Cephalon::load(currentdirpath.clone());

//Search the Index and database for results
let matches: Vec<Matches> = cephalon_knowledge_base.search(current_dir_path, query.query,5).unwrap();

//Iterate through matches and print them
for search_result in matches{
    println!("{}, {:?}",search_result.document_name, search_result.line);
}

} ```

Cephalon under the hood

Cephalon-rs is the base version of Cephalon purely written in Rust. It also uses other libraries such as serde, rayon, rust-bert, pdf-extract, minidom, and zip. It also uses clap to create the cli for Rust. For the index it uses the HNSW Index with default settings from hora-search.

Supported File Types