A queue is useful when a request should be accepted quickly but its full work takes longer, depends on a flaky system or needs controlled throughput. It also adds a second runtime, delayed outcomes and duplicate delivery. Small teams should add a queue to remove a specific bottleneck, not because every architecture diagram looks better with one.
Return a job ID and an explicit “accepted/processing” state, not a fake success implying the report, email or payment action already finished. Define what the caller can query, what happens on cancellation, and whether a failed job is automatically retried or needs operator review.
| Workload | Queue can help when... | Still define... |
|---|---|---|
| Webhook side effects | Provider expects a quick acknowledgment. | Signature verification, durable acceptance, dedupe and event ordering. |
| Reports or exports | Work takes seconds/minutes or needs larger resources. | Job visibility, cancellation, artifact expiry and progress. |
| External API sync | Partner limits or outages should not block user requests. | Backoff, rate budgets, freshness and reconciliation. |
| Notifications | Delivery can lag without changing the committed transaction. | Preference, dedupe, provider failures and user-visible status. |
Keep truly interactive operations synchronous when users need an immediate authoritative answer and the work reliably fits the request budget. A queue does not make an expensive algorithm faster; it changes who waits and how you observe the wait.
A common bug is committing an order, then crashing before publishing its message. The transactional outbox writes both business state and a pending event in one database transaction; a dispatcher publishes it and marks delivery. The consumer must still tolerate duplicates because a crash can happen after publish but before the dispatcher records success. For lower-risk systems, a simpler persisted job table may be enough; make its recovery scan explicit.
Many queue systems can deliver a message again, including after a worker times out or crashes. Give each logical operation a stable ID, enforce uniqueness or an idempotency record, and make external effects idempotent where possible. Acknowledge/delete only after durable completion. If the worker exceeds the visibility/lease period, extend it or split the job into bounded stages so another consumer does not process the same work concurrently.
Retry transient timeouts, throttling and temporary dependency failures with exponential backoff and jitter. Do not retry malformed input or permanent authorization failures endlessly. Cap attempts and elapsed time; move exhausted messages to a dead-letter or quarantine path with a reason and correlation data. Provide a safe replay tool that preserves the logical operation ID and records the new attempt.
Bound queue depth or age where possible and decide what users see when backlog grows. Scale consumers only while downstream systems can absorb the traffic. If arrival rate stays above service rate, adding workers can overload the database or partner API. Apply concurrency limits, batch size, per-tenant fairness and producer throttling. Queue age often tells users more than raw depth because message processing times vary.
Track accepted, queued, started, retried, completed and failed states with timestamps and correlation IDs. Alert on oldest-message age, retry rate, dead-letter growth, consumer errors and dependency throttling. Keep payloads minimal and sensitive data out of queue logs. Define retention for successful messages and job records separately from audit history.
A managed queue plus one worker and a job-status table is often enough. Avoid introducing a broker cluster, multiple queue technologies and custom orchestration before load or isolation needs justify them. Document ownership, deploy/rollback procedure, poison-message handling, replay authorization and what happens if the queue provider is unavailable.
Queues decouple acceptance from completion, but shift complexity into retries, duplicates, visibility and operations. Persist intent, publish reliably, process idempotently, bound retries and expose status. Add a queue where asynchronous work solves a measured user or reliability problem, and keep the operating model small enough for the team to own.