Deep Thought

This crate implements feedforward-neural Networks in rust.

A basix XOR training example might look like this: ```rust fn main() -> Result<()>{ // Build the input and label arrays let inputs = array![[0., 0.], [0., 1.], [1., 0.], [1., 1.],]; let labels = array![[0.], [1.], [1.], [0.]];

let dataset = Dataset::new(inputs, labels, 1., BatchSize::One)?;
let loss_fn = Loss::MSE;

// Build the neural net
let mut net = NeuralNetwork::new()
    .add_layer(Layer::new(2, 3).activation(Activation::Sigmoid))
    .add_layer(Layer::new(3, 3).activation(Activation::Sigmoid))
    .add_layer(Layer::new(3, 1).activation(Activation::Sigmoid));

let mut optim = SGD::new(&net).learning_rate(0.3).momentum(0.1);

// train the network
for epoch in 0..11000 {
    for (samples, labels) in dataset.iter_train() {
        let _out = net.forward(&samples);
        if epoch % 100 == 0 {
            println!("training epoch {}", epoch);
            println!(
                "  Loss: {}\n",
                &loss_fn.compute(&_out, &labels).mean().unwrap()
            );
        }
        net.backprop(samples, labels, &loss_fn, &mut optim);
    }
}

// evaluate the net
let mut total_loss: f64 = 0.;
// should ofc be iter_test but this dataset is kinda minimalistic
let test_iter = dataset.iter_train();
let num_test_samples = test_iter.num_batches * test_iter.batch_size;
for (sample, label) in test_iter {
    let out = net.forward(&sample);
    total_loss += loss_fn.compute(&out, &label).sum();
    println!("{} == {}", out.map(|&x| x.round()), label);
}

println!(
    "Mean loss over {} test samples: {:.2}",
    num_test_samples,
    total_loss / num_test_samples as f64
);
Ok(())

} ``` For more usage examples, please take a look at /examples

Features

• Linear Layers
• Common Activation functions like ReLU or Sigmoid
• Optional Serialization/Deserialization support via the serde feature

TODO

• Add decreasing learning rate (probably also easy)
• add CNN's

Find a array crate which satisfies the following conditions:

• Support basic stuff like max and argmax. I don't care that f64 doesn't implement Ord
• easier api than ndarray
• basically numpy but in rust and would be nice if gpu (:

Additional Ressources

Some stuff i found to be quite helpful if you are interested in understanding the math behind neural networks * Very nice article by Michael Nielsen