learnwell

Easy reinforcement learning framework, allowing you to quickly create Environments and test them.
Aims to be simple
Minimal external dependencies

Currently only Q-Learning is implemented

The state of this project is in alpha. Use at your own risk.

Getting started

See the taxi example and walk through the comments You can also run the examples for hike, taxi, mouse with: cargo run --release --example taxi

Imports: rust use learnwell::{ runner::Runner, agent::qlearning::QLearning, environment::{Environment, EnvironmentDisplay} strategy::decliningrandom::DecliningRandom, };

We then ask the Runner to run the agent for x number of epochs

Allows 2 modes: - Runner::run for normal operation - Runner::run_with_display to create a window and display image which gets updated as it runs

For example: rust Runner::run( QLearning::new(0.1, 0.98, DecliningRandom::new(epochs, 0.01)), //Agent TaxiEnvironment::default(), //Environment 400, //epochs ); or

rust Runner::run_with_display( QLearning::new(0.2, 0.99,DecliningRandom::new(epochs, 0.005) ), //Agent Hike::new(), //Environment 700_000, //epochs 10 //frames per second to refresh image );

We need:

We implement a few things to run

Environment

  1. State Struct - this is what we base our actions on
  2. Action (normally enum) - these are the actions we perform

  3. Environment Struct that implements the Environment<S,A> trait and depends on the State and Action. The Environment struct should hold the state, because we will refer to it later

    Agent

  4. the Agent algorithm (e.g. QLearning),

Implementation:

Note we derive Hash, Eq, PartialEq and Clone for both State and Action

State

```rust

[derive(Hash, Eq, PartialEq, Clone)]

pub struct TaxiState { taxi: Point, dropoff: Point, passenger: Point, in_taxi: bool, } ```

Action

```rust

[derive(Clone, Hash, PartialEq, Eq)]

pub enum TaxiAction { Up, Down, Left, Right, Dropoff, Pickup, } ```

Environment

rust pub struct TaxiEnvironment { state: TaxiState, //this is the actual state that gets saved in the qtable found: usize, //just a helper. there could be a few other items you want to track in the environment }

Status