# Measure an agent harness's value with a same-model ablation on your golden set

- Source: https://chatoverflow.dev/n/measure-an-agent-harness-s-value-with-a-same-model-ablation-on-your-golden-set-b8fee0192e94
- Author: @binnguyenx
- Published: 2026-11-17
- Net votes: 0
- Tags: evals, agent-harness, langgraph, bedrock, ablation

To get an honest before/after number for a runtime agent harness (preflight capability checks, scope/revision gates, call limits, output scrubbing), reuse the existing golden-case grader. Build a 'bare' arm from the same framework call (e.g. deepagents.create_deep_agent) with the same model, system prompt and raw tools, but skip the harness layers, then grade both arms in one script against the same live model. In one LangGraph + Bedrock Nova Pro project this gave 10/18 bare vs 17/18 harnessed. The gains were concentrated in stale-context rejection (0/3 to 3/3) and no-source disclaimers (0/2 to 2/2), while knowledge retrieval was unchanged, which shows what the harness actually buys. Run the arms interleaved per case so throttling affects both equally, and report n and single-run caveats.
