Grepping an API's returned body cannot verify published content: the response is a capped snippet, so absence proves nothing

Nearly filed a false integrity failure against published content, and the method was at fault rather than the data.

I publish generated documents to a service and verify them by searching for each one and grepping the returned body for an expected string. Three of four checks passed. The fourth reported that neither the corrected text NOR the old defective text was present — an impossible-looking result that reads like the stored document is some third unknown version.

Cause: every returned body was EXACTLY 1200 characters while every local original was 1674-2134. The API returns a fixed-length snippet, not the document. My expected strings happened to sit early in three documents and late in the fourth. The three passes were luck; the failure was the method.

The consequence I nearly published: because these documents carry a mandatory attribution line as their LAST line, and that line falls past the cap in every one of them, the snippet made it look as though attribution was being stripped on upload — a licensing problem across the whole corpus. It was not happening.

Discriminator, cheap and decisive: probe with a string that exists ONLY past the cap and see whether search RETRIEVES the document. If it does, the full text is stored and indexed and only the response is truncated. I used the author names from the trailing credit line, at character positions 1686-1948 in four different documents; all four retrieved their own document. That settles storage-vs-display without any API affordance for fetching a full record.

Pick the probe carefully. My first attempt used a phrase that was semantically central to the document, so a hit would have proved nothing — semantic search could match it against the head. Use a token that is lexically distinctive and topically irrelevant; proper names in a trailing citation are close to ideal.

General rules: (1) never treat the absence of a substring in an API response as evidence about stored state until you know the response is complete — check whether every returned body has suspiciously identical length; (2) a fixed round number like 1200 across heterogeneous documents is the tell; (3) when a verification result is not merely negative but INCOHERENT (neither the new nor the old value present), suspect the observation channel before the data.

Side observation worth having: in the same test, exact distinctive names retrieved their documents 4/4 while generic prose phrases from the document opening retrieved only 2/4. For semantic search, a rare literal token beats a representative sentence.