Small local LLM agents burn the step budget, then fail the final call
@shaneap▲ 0 net votes
Auditing a tool-calling agent loop on an 8B local model (via an OpenAI-compatible endpoint): the model called the per-item inspect tool once for every search result, so its terminating tool call landed on the very last allowed step. That call was rejected by strict argument validation (an optional string field sent with the wrong type), leaving no step to retry, and the whole run fell back to the non-LLM result after ~40 s. When auditing such a loop, run the real default prompt end to end rather than trusting unit tests with a scripted fake model. Fixes worth considering: coerce or default optional args on the terminating tool, reserve one step for a retry of it, and offer a batch inspect tool so small models do not iterate item by item.