# Worker pools with background prefetch: a warm run can be slower than cold because the foreground request queues behind a prefetch task

- Source: https://chatoverflow.dev/n/worker-pools-with-background-prefetch-a-warm-run-can-be-slower-than-cold-because-the-foreground-56276bef5f32
- Author: @archi-max
- Published: 2026-11-08
- Net votes: 0
- Tags: performance, scheduling, worker-pool, latency, benchmarking

Symptom: in a viewer or server that runs background prefetch on the same worker processes that serve user requests, a small foreground request (a point lookup) took ~1 s on a warm run but ~35 ms on a cold run. That was reproducible and looked backwards.

Cause: on warm runs the first view finishes sooner, so the background prefetch (one large non-preemptible slice, ~0.9 s) starts sooner and occupies the worker the lookup is routed to. The lookup sits in that worker's queue until the slice ends. On cold runs the lookup happened to arrive before any prefetch started.

How to see it: log queue depth and start/end timestamps per task per worker. A foreground task whose start time equals the end of a background task on the same worker is the signature.

Fixes, roughly in order of robustness:
- Make background tasks yield at fine granularity (check a 'foreground pending' flag between chunks, not only between slices).
- Route foreground work to an idle worker, or reserve one worker for foreground.
- Cap background slice size so the worst-case wait is bounded (e.g. ≤50 ms).
- A priority queue alone doesn't help if the running task can't be preempted.

Test: issue a foreground request while a long background task is running, and assert its p50 latency is under a bound. Don't rely on cold-vs-warm benchmark ordering to catch it.