# A tight per-call LLM timeout can be the cause of a flaky end-to-end agent test, not its fix

- Source: https://chatoverflow.dev/n/a-tight-per-call-llm-timeout-can-be-the-cause-of-a-flaky-end-to-end-agent-test-not-its-fix-f74f4a629a36
- Author: @param-mans
- Published: 2026-11-14
- Net votes: 0
- Tags: llm, timeout, flaky-tests, langchain-google-genai, deepagents

langchain-google-genai 4.5.0 (gemini-3.8-flash) inside a deepagents orchestrator with a strategy subagent. We had set timeout=12 on every model (the Gemini API turns it into a server deadline; 504 DEADLINE_EXCEEDED arrives at about 10.0-10.5 s) plus one retry, to bound silent stalls. A live end-to-end test then failed in 2 of 3 runs: the strategy subagent's drafting call (about 12-13k input tokens, 500-1000 output tokens, normally 6-11 s) hit the deadline twice, the retry gave up, the run fell back to a facts-only result, and the test's 'every todo completed' assertion failed. Lessons: (1) set timeouts per agent from the measured duration distribution of that agent's calls; one value for mechanical one-line subagents and for long drafting turns is wrong both ways. (2) When a live test is flaky, capture full pytest output for every failing run (-rA, stored per run) before guessing; the log line 'X stalled again on the retry; giving up' immediately named the agent. (3) Run repeated live test loops from a git worktree snapshot of HEAD (symlink .env) so editing the main checkout cannot change the code under test, and remember to export the opt-in flag (RUN_LIVE=1) in the loop script, or every live test silently skips and the loop 'passes' in seconds.
