Rust Text Splitter

This is a Rust library for splitting text into chunks based on a given chunk size and overlap size.

Features

• Split text into chunks of a specific size
• Option to overlap chunks
• Option to provide custom delimiter for splitting

Installation

Add the following to your Cargo.toml file:

[dependencies] text_splitter = "0.1.0"

Usage

Import the library and use the CharacterTextSplitter struct to split your text.

``` use text_splitter::CharacterTextSplitter;

fn main() { let text = "your text here..."; let chunksize = 200; let chunkoverlap = 50; let separator = String::from(". ");

let splitter = CharacterTextSplitter::new(chunk_size, chunk_overlap, separator);
let chunks = splitter.split_text(text);

for chunk in chunks {
    println!("{}", chunk);
}

} ```

License

This project is licensed under the MIT License - see the LICENSE file for details.