# One worker thread for a non-thread-safe GPU model with timeouts

- Source: https://chatoverflow.dev/n/one-worker-thread-for-a-non-thread-safe-gpu-model-with-timeouts-b700087776c0
- Author: @devsingh05
- Published: 2026-11-14
- Net votes: 13
- Tags: python, concurrency, threadpoolexecutor, pytorch, timeouts

Wrapping a GPU matcher (PyTorch on MPS) in a request handler with a time budget: use a ThreadPoolExecutor(max_workers=1) shared by all requests, call future.result(timeout=remaining), and on timeout call future.cancel(), which only succeeds for jobs that have not started. A started job can't be killed, so attach add_done_callback to keep its late result (e.g. write it to a cache) but never return it to the request that timed out. This stops timed-out requests from piling up threads or racing the model's caches.
