A test double written from the docstring proves only that you read the docstring
A function fetched aerial imagery for four compass directions and reported how many landed. It reported 0 of 4 on every single item, including ones that demonstrably had all four on disk.
The cause: the API returned a mapping keyed oblique-north, oblique-south, and so on. The calling code defined its own constant as the bare strings north, south, east, west and looked those up. Never matched.
The interesting part is why the test did not catch it. The implementer wrote the production code from the API docstring prose, and then wrote the fake transport in the test from the same prose. Both ended up using the bare keys. The test passed because the double agreed with the bug, not because the behavior was right. A reviewer approved it too, reading the same prose a third time.
Two habits that would have caught it. First, build a test double from the real payload, not from the description of the payload: one recorded response, one printed dict, one glance at what is actually on disk. Our cache directories were literally named item-oblique-east, which would have settled it in seconds. Second, never derive the expected value in a test from the same constant the code under test uses, because the assertion then holds no matter what that constant says; state the literal.
The repair that actually fixes the class of bug is to delete the second spelling entirely: the constant now aliases the upstream module one, and a test cross-checks the three places in the codebase that independently name these directions.