bevy_rl

bevy_rl is a tool for building Reinforcement Learning Gyms with Bevy game engine in Rust.

It lets you to build 3D AI environments to train your AI agents that learn from raw screen pixels.

Features

• REST API to control an agent
• Rendering to membuffer to get FirstPersonCamera pixels and feed to an agent

Usage

1. Define Action Space and Application State

```rust

[derive(Debug, Clone, Eq, PartialEq, Hash)]

enum AppState { InGame, // Actve state Control, // A paused state in which application waits for agent input Reset, // A request to reset environment state }

// List of possible agent actions (discrete variant) bitflags! { #[derive(Default)] pub struct PlayerActionFlags: u32 { const IDLE = 1 << 0; const FORWARD = 1 << 1; const BACKWARD = 1 << 2; const LEFT = 1 << 3; const RIGHT = 1 << 4; const TURNLEFT = 1 << 5; const TURNRIGHT = 1 << 6; const SHOOT = 1 << 7; } } ```

2. Enable AI Gym Plugin

```rust app // Plugin settings .insert_resource(AIGymSettings { \ viewport settings width: 768,
height: 768, })

// Actions
.insert_resource(Arc::new(Mutex::new(AIGymState::<PlayerActionFlags> { 
    ..Default::default()
})))

// Plugin
.add_plugin(AIGymPlugin::<PlayerActionFlags>::default());

```

3. Making sure environment is discreet

rust struct DelayedControlTimer(Timer);

```rust app.insertresource(DelayedControlTimer(Timer::fromseconds(0.1, true))); // 10 Hz app.addsystemset( SystemSet::onupdate(AppState::Control) // Game Systems .withsystem(turnbasedtextcontrolsystem) // System that parses user command .withsystem(executeresetrequest), // System that performs environment state reset );

app.addsystemset( SystemSet::onupdate(AppState::InGame) .withsystem(turnbasedcontrolsystem_switch), );

```

```rust fn turnbasedcontrolsystemswitch( mut appstate: ResMut>, time: Res, mut timer: ResMut, aigymstate: ResMut>>>, ) { if timer.0.tick(time.delta()).justfinished() { appstate.push(AppState::Control); physics_time.pause();

    let ai_gym_state = ai_gym_state.lock().unwrap();
    ai_gym_state.send_step_result(true);
}

} ```

4. Handling Agent Actions

```rust fn turnbasedtextcontrolsystem( aigymstate: ResMut>>>, mut appstate: ResMut>, ) { let mut aigymstate = aigymstate.lock().unwrap();

if !ai_gym_state.is_next_action() {
    return;
}

let unparsed_action = ai_gym_state.receive_action_string();

if unparsed_action == "" {
    ai_gym_state.send_step_result(false);
    return;
}

let action = match unparsed_action.as_str() {
    "FORWARD" => Some(PlayerActionFlags::FORWARD),
    "BACKWARD" => Some(PlayerActionFlags::BACKWARD),
    "LEFT" => Some(PlayerActionFlags::LEFT),
    "RIGHT" => Some(PlayerActionFlags::RIGHT),
    "TURN_LEFT" => Some(PlayerActionFlags::TURN_LEFT),
    "TURN_RIGHT" => Some(PlayerActionFlags::TURN_RIGHT),
    "SHOOT" => Some(PlayerActionFlags::SHOOT),
    _ => None,
};

if action.is_none() {
    ai_gym_state.send_step_result(false);
    return;
}

let player = player_query.iter().find(|e| e.name == "Player 1").unwrap();
{
    ai_gym_state.set_score(player.score as f32);
}

physics_time.resume();

control_player(
    action.unwrap(),
    player_movement_q,
    collision_events,
    event_gun_shot,
);

app_state.pop().unwrap();

} ```

5. Handling Environment Reset

```rust fn executeresetrequest( mut appstate: ResMut>, aigymstate: ResMut>>>, ) { let aigymstate = aigymstate.lock().unwrap(); if !aigymstate.isreset_request() { return; }

ai_gym_state.receive_reset_request();
app_state.set(AppState::Reset).unwrap();

} ```

Interacting with Environment

First Person camera pixels

GET http://localhost:7878/screen.png

Reset Environment

POST http://localhost:7878/reset

Perform Action

POST http://localhost:7878/step body=ACTION

Example usage

BevyStein is first-person shooter environment made with bevy_rl.

Limitations

bevy_rl is early stage of development and has following limitations:

1. Raw pixels are from GPU buffer and do not contain pixels from 2D camera
2. You must be careful with sending signals to step and reset requests or application can deadlock