A Rust implementation of Celery for producing and consuming asyncronous tasks with a distributed message queue.

• Send tasks from Python to Rust or vice versa ↔️
• Use provided brokers and backends (coming soon) or implement your own 🔧
• Natural async / await interface 🎇
• High performance and safe 🔥 🔥 🔥

Inspired by, well... Celery, but also batch-rs especially for it's awesome macro implementation.

Quick links

• API docs
• The Rusty Celery Book (work in progress)
• Library on crates.io
• Contributing

Getting started

The ./examples directory contains a simple Celery app that is implemented in both Rust (celeryapp.rs) and Python (celeryapp.py) using an AMQP broker.

Before running the examples you'll need to have the broker set up. RabbitMQ is a popular implementation of the AMQP protocol and provides a Docker image that you can run locally. Another option would be to use a hosting provider like CloudAMQP, which provides a free tier broker for development purposes.

Once you have your broker running, set the environment variable AMQP_ADDR to your broker URL. I also recommend setting RUST_LOG=debug. Then you can send a task from Python to Rust by running the script

bash ./examples/python_to_rust.sh

And send a task from Rust to Python by running

bash ./examples/rust_to_python.sh

Road map and current state

Functionality

| | Implemented | In-progress | | ------------- | ------------- | ------------ | | Core protocol | | | | Protocol enhancements | | |

Brokers

| | Status | Tracking | | ----- | ------ | -------- | | AMQP | | | | Redis | | |

Backends

| | Status | Tracking | | ----------- | ------ | -------- | | RPC (AMQP) | | | | Redis | | |

Team

Rusty Celery is an open source project backed by Structurely. Structurely is a start-up building customizable AI inside sales agents that has been using Celery in production back to circa 2017.