Skip to content
Gisila

Architecture

What it is

Section titled “What it is”

gisila_queue moves work out of the process that asks for it. Four roles take part:

  • Producer: any code that calls delay()/applyAsync()/send(). Usually your web server.
  • Broker: holds messages until a worker takes them. Redis, RabbitMQ, or an in-memory queue.
  • Worker: a process that reserves messages, runs the task handler, and records the outcome.
  • Result backend (optional): stores each task’s state and return value so a producer can read it.

Two optional roles sit beside them: Beat enqueues periodic tasks, and GisilaRose shows what the cluster is doing.

When to use it

Section titled “When to use it”
  • Work that would make a request slow: sending email, rendering PDFs, calling slow APIs.
  • Work that must survive a crash or a deploy: payments follow-ups, webhooks.
  • Work that runs on a schedule or after a delay.
  • Spreading CPU-heavy work across several processes or machines.

When not to use it

Section titled “When not to use it”
  • Work that must finish before you respond and takes milliseconds: just await it.
  • Fan-out inside one request with no durability needs: Future.wait is simpler.
  • Event streaming with replay and consumer groups: use a log (Kafka, Redis Streams). gisila_queue messages are consumed once and gone.

How it works

Section titled “How it works”
producer ──delay()──▶ GisilaQueue.send ──▶ Broker.enqueue
   │                    │ stores pending/scheduled          │ Redis: list, ZSET for ETA
   │                    ▼                                    │ RabbitMQ: queue, TTL buckets
   │               ResultBackend                             ▼
   │                    ▲                     Worker.reserve (unacked until done)
   │                    │                                    │
   │                    └──── store(started/success/failure) ◀ handler(ctx)
   │                                                         │ retry: enqueue copy with ETA
   │                                                         ▼
   │                                                    Broker.ack
   ▼
AsyncResult.get() ── polls ──▶ ResultBackend.fetch
  1. The producer validates the arguments as JSON, picks the queue, stores pending (or scheduled), and enqueues a TaskMessage. It gets an AsyncResult back immediately.
  2. A Worker reserves the message. On Redis and RabbitMQ the message stays recorded as in flight until acked.
  3. The worker looks up the task name in its registry, stores started, runs the handler, and stores success with the result, or schedules a retry, or stores failure and moves the message to the dead-letter queue.
  4. The worker acks. If it died before that, the broker hands the message to another worker.
  5. The producer, or anyone with the id, polls the backend through AsyncResult.

Every process builds the same GisilaQueue with the same broker settings and namespace. Producers only need the task names; workers need the task handlers registered. In practice you put registrations in one function both call.

Delays are the broker’s job: Redis keeps delayed messages in a sorted set and promotes them about once a second; RabbitMQ parks them in TTL queues. Retries are new messages with an ETA, so a retrying task doesn’t hold a worker.

Example

Section titled “Example”
import 'package:gisila_queue/gisila_queue.dart';


// lib/tasks.dart, imported by both processes
TaskRef<int> registerTasks(GisilaQueue app) =>
    app.task<int>('math.add', (ctx) async => (ctx.arg(0) as int) + (ctx.arg(1) as int));


// web process
final app = GisilaQueue.redis(namespace: 'shop');
final add = registerTasks(app);
final r = await add.delay([2, 3]);


// worker process
final workerApp = GisilaQueue.redis(namespace: 'shop');
registerTasks(workerApp);
await workerApp.worker(concurrency: 8).run();
Section titled “Related”