Do you need Redis to run background jobs?

Do you need Redis to run background jobs?

No. For most products a queue that runs on your existing database is the better choice: one less service to run, secure, back up, and pay for, and jobs that participate in the same transactions as your data. The historic reason to reach for a separate in-memory store was throughput, and the threshold where that actually matters is far higher than most products ever reach.

Here is the real tradeoff and what actually makes jobs reliable, which is not the queue you picked.

What background jobs are for

Anything that should not happen while somebody waits for a page:

• Sending email. A slow mail provider should never slow down a request.
• Calling other people's APIs, which can be slow or down.
• Processing uploads.
• Scheduled work: nightly cleanups, daily digests, expiring records.
• Anything that might need retrying, which is most things that touch a network.

The rule of thumb: if it can fail independently of the request, or take longer than the person is willing to wait, it belongs in a job.

The case for the database-backed queue

One less service. A separate queue store is a thing to install, configure, secure, monitor, back up, upgrade, and pay for. Removing it removes all of that at once, and at small scale it is a meaningful share of your operational surface.

Jobs and data stay consistent. This is the technically interesting advantage. If enqueuing a job is a write to the same database as the change that triggered it, both happen in the same transaction. Either the order is created and the confirmation email is queued, or neither is.

With a separate store you get the classic split-brain: the database write succeeds and the enqueue fails, so the order exists and nobody is told. Or the enqueue succeeds and the transaction rolls back, so a job runs for an order that does not exist. Both are avoidable, and avoiding them takes care you would rather spend elsewhere.

Jobs survive. They are rows. A restart does not lose them, and you can query them with the tools you already have.

You can inspect it. Debugging a stuck queue is a query, not a specialised client.

The honest case against

Throughput has a ceiling. A separate in-memory store handles far more jobs per second. If you are processing tens of thousands per minute, you will feel the difference.

Polling has a cost. A database queue checks for work periodically rather than being pushed to, which means a small constant load and a small latency floor. Neither is noticeable at ordinary volumes.

Very large backlogs are heavier. A queue table with millions of rows needs attention in a way a smaller one does not.

The honest summary: the crossover point is far above where most products live, and the operational saving is immediate. Start with the database, and if you ever genuinely outgrow it, you will have real numbers to make the change with.

The four rules that actually matter

The queue is the least important decision here. These are what determine whether your jobs work.

1. Jobs must be safe to run twice

The most important rule, and the most commonly broken.

Jobs get retried. A worker dies mid-run, a network call times out, a deploy interrupts things. A job that ran halfway and gets retried will run its first half again.

So "send the welcome email" must not send two. "Charge the customer" must not charge twice. The usual fixes are to record that the work was done and check that record first, or to write the operation so that doing it again changes nothing after the first time.

This is the same property payment webhooks need, for the same reason, and it is worth internalising once: anything that can be delivered more than once will be.

2. Pass ids, not objects

Enqueue the identifier and load the record inside the job.

If you serialize a whole object into the queue, the job runs against a snapshot from whenever it was enqueued, which may be minutes or hours stale. Worse, if the record was deleted in the meantime, you are operating on something that no longer exists. Load it fresh, and handle the case where it is gone.

3. Assume the job will fail

Not might. Will. Networks fail, providers have bad minutes, records disappear.

Configure retries with increasing delays, decide what happens after the last one, and make sure failures are visible somewhere you will actually look. A queue that silently drops work is worse than no queue, because you believe the work happened.

4. Scope jobs to their account explicitly

Easy to miss and important. A job runs with no request and no current account, so all the automatic scoping that protects your controllers does not apply.

Pass the account identifier into the job and scope every query inside it explicitly. This is one of the places tenant isolation usually gets missed, precisely because the surrounding machinery is absent.

Scheduled work

Recurring jobs, nightly cleanups, digests. Two things to watch.

Overlap. If a job runs every five minutes and one run takes six, you now have two running at once. Decide whether that is safe, and if not, prevent it.

Duplication across servers. If you ever run more than one application server, each will try to run the schedule. Make sure exactly one does.

The summary

• You do not need a separate service for background jobs. A database-backed queue is a real answer, not a compromise.
• It removes an entire operational dependency and keeps jobs consistent with your data.
• Reconsider only when you have measured throughput that genuinely needs more.
• The queue matters far less than the four rules: safe to run twice, pass ids, expect failure, scope explicitly.

Fewer moving parts is a feature. It is the same logic that makes one small server the right answer for longer than people expect.