Build a Simple Job Queue From Scratch in PHP (No Redis Required)

Every job queue tutorial reaches for Redis or a message broker before explaining what a queue is actually doing underneath — a table of pending work, a worker that claims and processes items, and a way to handle failures without losing anything. For a project that doesn't have Redis running yet, or one where adding a new piece of infrastructure isn't worth it for a modest volume of background jobs, a database table gets you most of the way there. Here's the whole thing built from scratch, including the race condition that let two workers grab and process the same job simultaneously.

The table: everything a queue needs to track

Schema::create('jobs_queue', function (Blueprint $table) {
    $table->id();
    $table->string('type');
    $table->json('payload');
    $table->unsignedTinyInteger('attempts')->default(0);
    $table->timestamp('available_at');
    $table->timestamp('reserved_at')->nullable();
    $table->timestamp('completed_at')->nullable();
    $table->text('last_error')->nullable();
    $table->timestamps();
});

available_at controls when a job is eligible to run, which is what makes delayed jobs and retry backoff possible without any extra machinery. reserved_at marks a job as claimed by a worker so a second worker doesn't pick it up too. That single column is where the interesting bug in this whole build lives.

Enqueuing a job

function enqueue(string $type, array $payload, int $delaySeconds = 0): void
{
    DB::table('jobs_queue')->insert([
        'type' => $type,
        'payload' => json_encode($payload),
        'available_at' => now()->addSeconds($delaySeconds),
        'created_at' => now(),
        'updated_at' => now(),
    ]);
}

enqueue('send_welcome_email', ['user_id' => 42]);
enqueue('cleanup_expired_sessions', [], delaySeconds: 300);

Nothing surprising here — a row goes in, and available_at defaults to right now unless a delay is specified. The real work happens on the worker side.

The first worker: naive, and broken under concurrency

function processNextJob(): void
{
    $job = DB::table('jobs_queue')
        ->whereNull('reserved_at')
        ->where('available_at', '<=', now())
        ->orderBy('id')
        ->first();

    if (! $job) {
        return;
    }

    DB::table('jobs_queue')->where('id', $job->id)->update(['reserved_at' => now()]);

    runJob($job);
}

This looks correct read top to bottom, and it worked fine in every manual test — running processNextJob() in a single script, one call at a time. It broke the moment we ran two worker processes at once for higher throughput. Both workers could execute the SELECT in the first block within microseconds of each other, before either had updated reserved_at — meaning both selected the exact same unreserved job, and both proceeded to mark it reserved and run it, resulting in the same job (in our case, sending a welcome email) executing twice for the same user.

Why this is a classic check-then-act race condition

The bug has a name: check-then-act. Between the SELECT that checks "is this job unreserved" and the UPDATE that acts on that check by reserving it, there's a window where another process can run the exact same check and get the exact same answer, because nothing has changed in the database yet to reflect the first process's intent. The gap is measured in milliseconds, but two worker processes polling on a tight loop hit that window often enough to matter in practice, not just in theory.

The fix: an atomic claim in a single query

The fix is combining the check and the act into one atomic database operation, so there's no window between them for a second process to interleave:

function claimNextJob(): ?object
{
    return DB::transaction(function () {
        $job = DB::table('jobs_queue')
            ->whereNull('reserved_at')
            ->where('available_at', '<=', now())
            ->orderBy('id')
            ->lockForUpdate()
            ->first();

        if (! $job) {
            return null;
        }

        DB::table('jobs_queue')->where('id', $job->id)->update(['reserved_at' => now()]);

        return $job;
    });
}

lockForUpdate() acquires a row-level lock on the selected row for the duration of the transaction, which means a second worker's identical query, running concurrently, blocks until the first worker's transaction commits — and by the time it unblocks, reserved_at is already set, so the second worker's WHERE reserved_at IS NULL condition no longer matches that row at all. Wrapping the select-and-update in a single transaction with a row lock closes the exact gap the naive version left open.

Handling failures without losing the job

function runJob(object $job): void
{
    try {
        $handler = resolveHandler($job->type);
        $handler(json_decode($job->payload, true));

        DB::table('jobs_queue')->where('id', $job->id)->update(['completed_at' => now()]);
    } catch (\Throwable $e) {
        $attempts = $job->attempts + 1;
        $backoffSeconds = min(300, 5 * (2 ** $attempts)); // exponential backoff, capped at 5 minutes

        DB::table('jobs_queue')->where('id', $job->id)->update([
            'attempts' => $attempts,
            'reserved_at' => null, // release it back to the pool
            'available_at' => now()->addSeconds($backoffSeconds),
            'last_error' => $e->getMessage(),
        ]);
    }
}

Releasing reserved_at back to null on failure, combined with pushing available_at into the future using exponential backoff, means a failing job gets retried automatically with increasing delay between attempts rather than either being lost entirely or hammered immediately in a tight failure loop that could make a transient problem worse.

A second bug: jobs stuck reserved forever after a worker crash

The transaction-and-lock fix solved concurrent double-processing. It didn't solve a related problem that showed up separately: if a worker process crashed or got killed mid-job — a deploy restarting the server, an out-of-memory kill — the job it had claimed stayed marked as reserved_at forever, since nothing ever ran the failure-handling code to release it. That job would sit invisible to every future worker poll indefinitely, silently never processed and never retried.

function releaseStaleReservations(): void
{
    // A job reserved more than 10 minutes ago with no completion
    // was almost certainly abandoned by a crashed worker
    DB::table('jobs_queue')
        ->whereNotNull('reserved_at')
        ->whereNull('completed_at')
        ->where('reserved_at', '<', now()->subMinutes(10))
        ->update(['reserved_at' => null]);
}

Running this as a separate scheduled check every few minutes — independent of the main worker loop — catches jobs abandoned by a crashed process and puts them back in the eligible pool. The 10-minute threshold is a judgment call specific to how long our longest legitimate job normally takes; setting it too short risks releasing a job that's still genuinely running and creating a new double-processing scenario, which is worth tuning against your own actual job durations rather than copying a round number.

What this doesn't replace

• High-throughput queues. Polling a database table works well into the low thousands of jobs per minute on modest hardware, but a dedicated queue system with push-based delivery scales further with less database load at genuinely high volume.
• Complex routing and priority queues. This version processes jobs roughly in order with no priority tiers — adding that is possible but starts adding real complexity a dedicated system already solved.
• Cross-service queuing. A database table is a fine fit when the producer and consumer share the same database. Once multiple independent services need to enqueue and consume jobs across different databases, a dedicated broker earns its complexity.

For a background job system inside a single application with moderate volume, this covers the real requirements — enqueue, claim safely under concurrency, retry with backoff, and recover from a crashed worker — without adding a new piece of infrastructure to operate. If your jobs need to notify an external system on completion, pairing this with proper retry handling on the receiving end is worth reading in webhook retry logic that doesn't duplicate data, and if job payloads need bounded, expiring storage rather than living in the queue table indefinitely, building a simple in-memory cache with expiry covers the same expiration logic applied to a different problem.

Running the worker loop and knowing when to actually poll

Everything above handles claiming and processing a single job. The worker itself needs a loop that repeatedly claims and runs jobs, and the naive version of that loop — polling as fast as possible in a tight while (true) — burns CPU checking an empty table hundreds of times a second during quiet periods:

function runWorker(): void
{
    while (true) {
        releaseStaleReservations();

        $job = claimNextJob();

        if ($job === null) {
            sleep(2); // nothing to do — back off instead of hammering the table
            continue;
        }

        runJob($job);
    }
}

A short, fixed sleep when the queue is empty is a reasonable default for moderate volume — it caps the worst-case delay between a job becoming available and a worker picking it up at roughly the sleep duration, while keeping the idle polling load negligible. For workloads where that delay actually matters, a short-lived database listen/notify mechanism (Postgres's LISTEN/NOTIFY, for instance) can wake a worker immediately on insert rather than waiting for the next poll — a reasonable next step once the fixed-interval version's latency becomes a real, measured problem rather than a theoretical one.

Running more than one worker safely

The atomic claim with lockForUpdate() is exactly what makes running several worker processes concurrently safe — each one calls claimNextJob() independently, and the row lock ensures no two of them ever walk away with the same job. In practice, running two or three worker processes against the same table, each in its own loop, scaled throughput close to linearly for us up to a handful of processes, before database connection overhead started being the actual bottleneck rather than anything in the queue logic itself.

# Running multiple workers as separate processes, e.g. via Supervisor
[program:job-worker]
command=php artisan queue:run-worker
numprocs=3
process_name=%(program_name)s_%(process_num)02d
autostart=true
autorestart=true

Checking on the queue's health

A queue with no visibility into its own backlog is easy to lose track of — jobs can quietly pile up faster than workers process them, and the first sign of trouble is often a user complaint about a delayed email, not an alert. A simple health check comparing the count of unprocessed jobs against a threshold, checked on a schedule, catches this before it becomes a user-visible problem:

function checkQueueBacklog(): void
{
    $pending = DB::table('jobs_queue')
        ->whereNull('completed_at')
        ->where('available_at', '<=', now())
        ->count();

    if ($pending > 500) {
        Log::warning("Job queue backlog is unusually high: {$pending} pending jobs.");
    }
}

The threshold itself should reflect your actual normal volume — 500 might be alarming for a low-traffic app and completely unremarkable for a high-traffic one, so it's worth setting based on a few weeks of observed normal backlog rather than picking a round number without checking first.

What we'd tell someone building this for the first time

• Test the race condition deliberately, not by hoping it shows up in normal use. Running two worker processes against a seeded batch of jobs in a test environment surfaces the double-processing bug in seconds, far faster than waiting for it to happen in production under real concurrent load.
• Build the stale-reservation cleanup before you need it, not after a crashed deploy leaves jobs stuck. It's a small addition and the failure mode it prevents is silent and easy to miss until someone notices a specific job that never ran.
• Log every failure with enough detail to actually debug it later — last_error as a plain text column is a small addition that pays for itself the first time a job is failing repeatedly and nobody can tell why without it.

The decision that mattered more than any of the code above

Every specific fix in this walkthrough — the atomic claim, the stale-reservation sweep, the backoff on retry — addresses a failure mode that only becomes visible once a queue is under real, concurrent load. None of them were obvious from reading the naive first version in isolation, and that's worth naming honestly rather than presenting this build as if the final version was the plan from the start. The naive version genuinely looked complete on a first read: it enqueues, it claims, it runs. What made the difference wasn't writing more careful code from the outset, it was deliberately testing under the specific condition — two workers running concurrently against the same table — that the naive version was never actually tested against, because a single manual test run in a terminal never exercises concurrency at all.

That's a genuinely useful habit to carry into other queue-like systems generally, not just this one: any system with a "claim work from a shared pool" step has this exact same category of race condition waiting to happen, whether it's built on a database table, a Redis list, or a dedicated broker, and the fix is always some version of the same idea — combine the check and the claim into one atomic operation, rather than trusting that the gap between two separate operations will never matter in practice. It matters faster than most people expect once real concurrency shows up.

Worth being specific about what "moderate volume" actually means in practice for this design, since that's the qualifier the whole approach depends on. On ordinary hardware, a single Postgres or MySQL table handling a poll-and-claim pattern comfortably supports a few hundred jobs per minute across a handful of worker processes before database connection overhead and lock contention become the limiting factor rather than anything specific to the queue logic itself. That ceiling is high enough to cover the actual background-job needs of most applications — sending emails, processing uploads, running scheduled cleanup — without ever requiring a dedicated broker at all. It's a ceiling worth knowing exists, not a reason to avoid this approach until you've actually measured your own workload against it.