🏋️‍♀️ bevy_rl

🏗️ Build 🤔 Reinforcement Learning 🏋🏿‍♂️ Gym environments with 🕊 Bevy engine to train 👾 AI agents that 💡 can learn from 📺 screen pixels or defined obeservation state.

Compatibility

| bevy version | bevy_rl version | | ------------ | :-------------: | | 0.7 | 0.0.5 | | 0.8 | 0.8.4 | | 0.9 | 0.9.8-beta |

📝Features

• Set of APIs to implement OpenAI Gym interface
• REST API to control an agent
• Rendering to RAM membuffer

👩‍💻 Usage

1. Define Action and Obeservation Space

Observation space needs to be Serializable for REST API to work.

```rust // Action space

[derive(Default)]

pub struct Actions { }

// Observation space

[derive(Default, Serialize, Clone)]

pub struct State { } ```

2. Enable AI Gym Plugin

Width and height should exceed 256, otherwise wgpu will panic.

rust // Setup bevy_rl let ai_gym_state = AIGymState::<Actions, State>::new(AIGymSettings { width: u32, // Width and height of the screen height: u32, // ... num_agents: 1, // Number of agents — each will get a camera handle render_to_buffer: false, // You can disable rendering to buffer pause_interval: 0.01, // 100 Hz ..default() }); app.insert_resource(ai_gym_state) .add_plugin(AIGymPlugin::<Actions, State>::default());

2.1 (Optional) Enable Rendering to Buffer

If your environment exports raw pixels, you will need to attach a render target to each camera of your agents.

```rust pub(crate) fn spawncameras( aigymstate: Res>, ) { let mut aigymstate = aigymstate.lock().unwrap(); let aigymsettings = aigym_state.settings.clone();

for i in 0..ai_gym_settings.num_agents {
    let render_image_handle = ai_gym_state.render_image_handles[i as usize].clone();
    let render_target = RenderTarget::Image(render_image_handle);
    let camera_bundle = Camera3dBundle {
        camera: Camera {
            target: render_target,  // Render target is baked in bevy_rl and used to export pixels
            priority: -1,           // set to -1 to render at the firstmost pass
            ..default()
        },
        ..default()
    };
    commands.spawn(camera_bundle);
}

} ```

4. Handle bevy_rl events

| Event | Description | | ------------------ | ------------------------------------- | | EventReset | Reset environment to initial state | | EventControl | Switch to control state | | EventPauseResume | Pause or resume environment execution |

```rust // EventPauseResume fn bevyrlpauserequest( mut pauseeventreader: EventReader, aigymstate: Res>, ) { for _ in pauseeventreader.iter() { // Pause simulation (physics engine) // ... // Collect state into serializable struct let envstate = EnvironmentState(...); // Set bevyrl gym state let mut aigymstate = aigymstate.lock().unwrap(); aigymstate.setenvstate(envstate); } }

// EventControl fn bevyrlcontrolrequest( mut pauseeventreader: EventReader, mut simulationstate: ResMut>, ) { for control in pauseeventreader.iter() { let unparsedactions = &control.0; for i in 0..unparsedactions.len() { if let Some(unparsedaction) = unparsedactions[i].clone() { let action: Vec = serdejson::fromstr(&unparsedaction).unwrap(); // Pass control inputs to your agents // ... } } // Resume simulation (physics engine) // ... // Switch to SimulationState::Running state of bevyrl simulation_state.set(SimulationState::Running); } } ```

Register systems to handle bevy_rl events.

rust // bevy_rl events app.add_system(bevy_rl_pause_request); app.add_system(bevy_rl_control_request);

💻 AIGymState API

| Method | Description | | -------------------------------------------------- | ----------------------------------- | | set_reward(agent_index: usize, score: f32) | Set reward for an agent | | set_terminated(agent_index: usize, result: bool) | Set termination status for an agent | | reset() | Reset bevy_rl state | | set_env_state(state: State) | Set current environment state |

🌐 REST API

| Method | Verb | bevy_rl version | | ----------------- | -------- | --------------------------------------------- | | Camera Pixels | GET | http://localhost:7878/visual_observations | | State | GET | http://localhost:7878/state | | Reset Environment | POST | http://localhost:7878/reset | | Step | GET | http://localhost:7878/step payload=ACTION |

✍️ Examples

• bevyrlshooter — example FPS project
• bevyquadrupedneuralcontrol — quadruped locomotion with bevymujoco and bevy_rl