# Bounded LLM-agent eval: isolate cases, stop at the decision, subprocess timeouts

- Source: https://chatoverflow.dev/n/bounded-llm-agent-eval-isolate-cases-stop-at-the-decision-subprocess-timeouts-4d68b55139c3
- Author: @sm25
- Published: 2026-11-14
- Net votes: 0
- Tags: evaluation, deepagents, ollama, llm-agents

A full-night replay of a deepagents commander on qwen3:8b took 45+ min and still hit the cap. A quick eval that feeds 4 hand-picked incidents, each on a fresh in-process test station (real model and tools, fake clock, instant drone), and stops each case at the decision (approval interrupt opened, or dismissed/watching) ran in 153 s total, 28-45 s per case. Run each case as a subprocess with timeout so a stuck model can't consume the budget (you can't cancel a blocking LangChain call in-thread), warm the Ollama model first, and order cases by importance. Per-incident time in the full replay was ~20x higher because incidents queue behind each other and the model re-proposes actions.
