Corgi

A neural network, and tensor dynamic automatic differentiation implementation for Rust.

```rust for _ in 0..iterations { // array operations are never in-place for corgi, so values never change let input = Array::from((vec![batchsize, inputsize], vec![...])); let target = Array::from((vec![batchsize, outputsize], vec![...]));

let _result = model.forward(input);
let loss = model.backward(target);
// update the parameters, and clear gradients (backward pass only sets gradients)
model.update();

println!("loss: {}", loss);

} ```

Design

Array operations are never in-place, meaning array values are never modified.
Eager execution.
Dynamic-as-possible computational graph. ```rust for _ in 0..10 { c = &c + &(&a * &b); if c[0] > 50.0 { c = &c * &a; } }

c.backward(None); ``* The Array is responsible differentiates operations done on it for the backward pass. * No graph structure for ergonomics - anArray` contains only its children. * Arrays do note store consumers (at the moment). They store consumer counts instead.

BLAS

The openblas, or netlib features can be enabled.
Versions prior to 0.9.7 of Corgi did not prioritise optimisation, and will be slow.

Tracked Arrays

Arrays are untracked by default, so if gradients are required, tracked(), or start_tracking() must be used (see the documentation for details).
Tracked arrays are arrays which require gradients to be computed, and stored.
For more information, see the documentation for tracked(), and untracked() in array.rs.

Examples

Fully-connected neural network (full version): ```rust let initializer = initializer::he(); let relu = activation::relu(); let softmax = activation::softmax(); let ce = cost::crossentropy(); let gd = GradientDescent::new(learningrate); let l1 = Dense::new(inputsize, hiddensize, &initializer, Some(&relu)); let l2 = Dense::new(hiddensize, outputsize, &initializer, Some(&softmax)); let mut model = Model::new(vec![&mut l1, &mut l2], &gd, &ce);

for _ in 0..iterations { let mut input = vec![0.0; inputsize * batchsize]; let mut target = vec![0.0; outputsize * batchsize];

// set inputs, and targets

// arrays in corgi should not be mutated after creation, so we initialise the values first
let input = Array::from((vec![batch_size, input_size], input));
let target = Array::from((vec![batch_size, output_size], target));

let _result = model.forward(input);
let loss = model.backward(target);
// update the parameters, and clear gradients (backward pass only sets gradients)
model.update();

println!("loss: {}", loss);

} * Dynamic computational graph:rust let a = arr![5.0].tracked(); let b = arr![2.0].tracked(); let mut c = arr![0.0].tracked();

for _ in 0..10 { c = &c + &(&a * &b); if c[0] > 50.0 { c = &c * &a; } }

assert_eq!(c, arr![195300.0]);

c.backward(None); asserteq!(c.gradient(), arr![1.0]); asserteq!(b.gradient(), arr![97650.0]); assert_eq!(a.gradient(), arr![232420.0]); ``` * Custom operation (still needs some work).

Resources

Shields are from shields.io.
MIT 6.034 on OpenCourseWare for a primer on Backward Propagation.
CS231n YouTube recordings for a primer on Convolutional Neural Networks.

A lot of the library was built around being as dynamic as possible, meaning if chosen well, some design choices might be similar to other dynamic computational graph libraries.