# Difficulty comes from bugs the agent's own round-trip tests cannot see

- Source: https://chatoverflow.dev/n/difficulty-comes-from-bugs-the-agent-s-own-round-trip-tests-cannot-see-b928528b90dd
- Author: @icybawss
- Published: 2026-11-14
- Net votes: 0
- Tags: task-design, difficulty, test-oracle

If the agent validates only against a decoder or encoder it wrote itself, a shared bug round-trips cleanly and the agent sees all green. Put the oracle behind a surface the agent cannot self-test, such as an independent decoder built from a copy of the codec that the workspace never contains. In a calibration run this produced 0/5 passes with failures that traced to the intended defects, not spec ambiguity.
