learnwell

Easy reinforcement learning framework, allowing you to quickly create Environments and test them.
Aims to be simple
Minimal external dependencies Framework to create your own implementations

Implementation examples - Q-Learning - Deep Q Learning (DQN)

The state of this project is in alpha. Use at your own risk.

Getting started

See the taxi example and walk through the comments cargo run --release --example taxi you can also run the following examples:

• hike - runs with display
• taxi
• mouse
• mouseimage - DQN
• taxiimage - DQN, runs with display

Imports: rust use learnwell::{ runner::Runner, agent::qlearning::QLearning, environment::{Environment, EnvironmentDisplay} strategy::decliningrandom::DecliningRandom, };

We then ask the Runner to run the agent for x number of epochs

Allows 2 modes: - Runner::run for normal operation - Runner::run_with_display to create a window and display image which gets updated as it runs

For example: rust Runner::run( QLearning::new(0.1, 0.98, DecliningRandom::new(epochs, 0.01)), //Agent TaxiEnvironment::default(), //Environment 400, //epochs ); or

rust Runner::run_with_display( QLearning::new(0.2, 0.99,DecliningRandom::new(epochs, 0.005) ), //Agent Hike::new(), //Environment 700_000, //epochs 10 //frames per second to refresh image );

We need:

• Environment - this is the game/scenario we want to learn
• Agent - this is what interacts with the environment

We implement a few things to run

Environment

1. State Struct - this is what we base our actions on
2. Action (normally enum) - these are the actions we perform

3. Environment Struct that implements the Environment<S,A> trait and depends on the State and Action. The Environment struct should hold the state, because we will refer to it later

   Agent

4. the Agent algorithm (e.g. QLearning),

Implementation:

Note we derive Hash, Eq, PartialEq and Clone for both State and Action

State

```rust

[derive(Hash, Eq, PartialEq, Clone)]

pub struct TaxiState { taxi: Point, dropoff: Point, passenger: Point, in_taxi: bool, } ```

Action

```rust

[derive(Clone, Hash, PartialEq, Eq)]

pub enum TaxiAction { Up, Down, Left, Right, Dropoff, Pickup, } ```

Environment

rust pub struct TaxiEnvironment { state: TaxiState, //this is the actual state that gets saved in the qtable found: usize, //just a helper. there could be a few other items you want to track in the environment }

Status

• [X] implement Qlearning
• [X] implement deep qlearning
• [ ] move optional functionality to features (e.g. display, fxhasher)