Effectum

A Rust job queue library, based on SQLite so it doesn't depend on any other services.

Currently this is just a library embeddable into Rust applications, but future goals include bindings into other languages and the ability to run as a standalone server, accessible by HTTP and gRPC. This will be designed so that a product can start with the embedded version to use minimal infrastructure, and then move to the server version with minimal changes when the time comes to scale out.

```rust use effectum::{Error, Job, JobState, JobRunner, RunningJob, Queue, Worker};

[derive(Debug)]

pub struct JobContext { // database pool or other things here }

[derive(Serialize, Deserialize)]

struct RemindMePayload { email: String, message: String, }

async fn remindmejob(job: RunningJob, context: Arc) -> Result<(), Error> { let payload: RemindMePayload = job.json_payload()?; // do something with the job Ok(()) }

[tokio::main(flavor = "current_thread")]

async fn main() -> Result<(), Error> { // Create a queue let queue = Queue::new(&PathBuf::from("effectum.db")).await?;

// Define a type job for the queue. let ajob = JobRunner::builder("remindme", remindmejob).build();

let context = Arc::new(JobContext{ // database pool or other things here });

// Create a worker to run jobs. let worker = Worker::builder(&queue, context) .maxconcurrency(10) .jobs([ajob]) .build();

// Submit a job to the queue. let jobid = Job::builder("remindme") .runat(time::OffsetDateTime::nowutc() + std::time::Duration::fromsecs(3600)) .jsonpayload(&RemindMePayload { email: "me@example.com".tostring(), message: "Time to go!".tostring() })? .add_to(&queue) .await?;

// See what's happening with the job. let status = queue.getjobstatus(jobid).await?; asserteq!(status.state, JobState::Pending);

// Do other stuff...

Ok(()) } ```

Changelog

Full Development Notes

Roadmap

0.1

• Multiple job types
• Jobs can be added with higher priority to "skip the line"
• Workers can run multiple jobs concurrently
• Schedule jobs in the future
• Automatically retry failed jobs, with exponential backoff
• Checkpoints to allow smart resumption of a job if it fails midway through.
• Immediately schedule a retry for jobs that were running when the process restarts unexpectedly

Soon

• Optional sweeper to prevent "done" job data from building up indefinitely
• Create recurring scheduled jobs
• Cancel or modify pending jobs

Later

• Node.js bindings
• Run as a standalone server over HTTP or gRPC
• Helpers for communicating with the queue via the outbox pattern.