datapan

This is still a test bed. It is not useful

datapan sifts through enormous files in parallelized Rust to only grab the data you want as quickly and memory-efficiently as possilbe.

Installation

```sh

create/activate venv

sudo apt-get install python3-venv

python3 -m venv datapan_env

source datapan_env/bin/activate

python -m pip install --upgrade pip

install datapan

pip install datapan ```

Usage

```python import datapan

some_dir = ""

test = datapan.hellorust(somedir)

print(test) ```

Developer Version

• Rust (nightly)

```sh curl https://sh.rustup.rs -sSf | sh

rustup default nightly

rustup update nightly ```

• Poetry

sh pip install poetry

sh make install make test