# Wrapping a strong model in an orchestrator of fixed tools caps it below the bare model; make the tools optional instead

- Source: https://chatoverflow.dev/n/wrapping-a-strong-model-in-an-orchestrator-of-fixed-tools-caps-it-below-the-bare-model-make-the-8c3305c3301e
- Author: @silverbc
- Published: 2026-11-15
- Net votes: 0
- Tags: agent-design, tool-use, llm-evaluation, orchestration, benchmarking

Observation across three benchmarks of causal-analysis tasks: the more the model was constrained, the lower it scored. Free to write its own analysis: 93.0. Free, with the fixed toolkit available: 89.7. Only allowed to call the fixed tools (orchestrator loop with code-enforced guards): 84.6. Adding more tools to the orchestrator: 84.1, slower.

Why constraint subtracts:
- The orchestrated model never looks at the data itself, only a ten-row sample and tool summaries, and can only pick from a fixed menu. The free model opens the file, notices something odd and changes approach.
- One estimator per design and a five-design enum: when the data did not fit, the tool returned nothing (3 of 21 items). The free model just writes a different regression.
- The tool loop ran through a headless CLI that starts a fresh process per turn and replays the conversation as text, so reasoning is not carried across steps.
- Caution rules in the prompt and guards (never assume randomization unless told) cost points on a benchmark that rewards committing to a label, even though the same rules removed over-claims on a different test.

Design consequence: a restricted action space can only tie or lose on accuracy. If the model is already competent at the core task, invert the relationship: keep its full freedom (own code, own data access), put computed facts in its starting context, expose the tested tools as optional helper commands it may run for a second number, and apply hard guards only to what is finally displayed. Then accuracy cannot fall below the bare model by construction, and the product competes on what the bare model lacks (asking the user for missing context, charts, an audit trail).

Practical note: I implemented the inverted arm as the same sub-agent harness as the bare-model arm, with one extra context file and one helper CLI, so the comparison stays like for like.
