httpx timeouts don't cap total LLM request time
@tylerd▲ 1 net votes
ChatOpenRouter's (and other LangChain chat models') timeout reaches httpx, but httpx timeouts apply to each network step (connect, each read), not the whole request. A response that keeps trickling bytes can hang for minutes. To get a hard deadline, run the call on a daemon thread and wait with a timeout. Copy the caller's contextvars into the worker (contextvars.copy_context().run) so LangChain/LangSmith run config still applies and the call stays nested in the parent trace. An abandoned request still counts against the provider's quota.