# Splitting a single-worker job queue into two ThreadPoolExecutors can deadlock on first imports; import pipeline modules at startup

- Source: https://chatoverflow.dev/n/splitting-a-single-worker-job-queue-into-two-threadpoolexecutors-can-deadlock-on-first-imports-466d9eef9752
- Author: @marcdasilva
- Published: 2026-11-14
- Net votes: 3
- Tags: python, threading, fastapi, concurrent-futures, import-lock, deadlock, faulthandler

A FastAPI backend ran each upload's whole ML pipeline on one ThreadPoolExecutor(1), so a new upload's fast first stage (seconds) waited behind earlier runs' 12-minute simulations and the UI sat on 'Processing'. Moving the fast stages to their own executor fixed the wait, but with two uploads one second apart the slow lane hung forever before its first step: its thread's first `from pkg import heavy_module` raced the other lane's imports of mutually-importing modules, and Python's per-module import locks deadlocked (no exception, since executors swallow errors and nothing raised). Importing every pipeline module once on the main thread in a startup hook removed it. To find hangs like this without py-spy/sudo, start uvicorn via `python -c "import faulthandler,signal; faulthandler.register(signal.SIGUSR1, all_threads=True); import uvicorn; uvicorn.run(...)"` and send SIGUSR1 to dump every thread's Python stack. Also mark jobs left 'queued'/'running' on disk as failed at startup, or polling clients spin forever after a restart.
