ggblas

ggblas is a library aimed to provide a simple and ergonomic access to the matrixmuliplication implemented in ggml

This library adds on top a threadpool with the physical number of cores each thread being pinned to their respective counterpart.

Usage

```rust use ggblas::batched_sgemm;

let a = vec![1., 2., 3., 4.]; let b = vec![1., 2., 3., 4.]; let mut c = vec![0., 0., 0., 0.];

// Simple (2, 2) x (2, 2) batchedsgemm(&a, &b, &mut c, 2, 2, 2); asserteq!(c, &[7., 10., 15., 22.]);

// Different shape (1, 4), (4, 1) let mut c = vec![0.]; batchedsgemm(&a, &b, &mut c, 1, 1, 4); asserteq!(c, &[30.]);

// batched (2, 2, 1), (2, 1, 2) // batching is done implicitly let mut c = vec![0., 0., 0., 0., 0., 0., 0., 0.]; batchedsgemm(&a, &b, &mut c, 2, 2, 1); asserteq!(c, &[1.0, 2.0, 2.0, 4.0, 9.0, 12.0, 12.0, 16.0]); ```

Performance

Current performance can be see here

Intel

i5-9300 (avx2)

```bash test benchggblasn ... bench: 469,739 ns/iter (+/- 3,111) test benchggblast ... bench: 317,049 ns/iter (+/- 5,450) test benchmkln ... bench: 140,561 ns/iter (+/- 1,095) test benchmklt ... bench: 185,928 ns/iter (+/- 2,781)

(cblas)

test benchblasn ... bench: 5,955,545 ns/iter (+/- 87,172) test benchblast ... bench: 10,153,008 ns/iter (+/- 528,645)

(matrixmultiply+threading)

test benchmatrixmultiplyn ... bench: 869,372 ns/iter (+/- 205,883) test benchmatrixmultiplyt ... bench: 841,705 ns/iter (+/- 12,706) ```

M1 (neon)

bash test bench_ggml_n ... bench: 640,552 ns/iter (+/- 21,558) test bench_ggml_t ... bench: 270,919 ns/iter (+/- 10,761) test bench_matrixmultiply_n ... bench: 944,152 ns/iter (+/- 38,737) test bench_matrixmultiply_t ... bench: 809,709 ns/iter (+/- 13,350) test bench_blas_n ... bench: 97,389 ns/iter (+/- 701) test bench_blas_t ... bench: 628,720 ns/iter (+/- 87,855)

License: Apache-2.0