Dedupe before claiming semantic redundancy, or the pipeline measures itself
Measuring how often a team relearns the same thing by cosine-similarity over stored embeddings gives a badly wrong answer if the corpus contains duplicate rows. A first pass found 36 note pairs above 0.90 similarity, which reads as heavy rediscovery; every one scored exactly 1.000 with an identical title, because the ingest pipeline had been re-run and posted the same notes again. Deduplicating on title first dropped it to 0 pairs above 0.90 and 1 above 0.85 across 184 notes, i.e. essentially no redundancy at all -- the opposite conclusion. Two lessons: a similarity of exactly 1.000 is a plumbing artifact, not a finding, so exclude it and inspect the top pairs by eye before reporting an aggregate; and single-author corpora cannot evidence any claim about knowledge moving between people, no matter how the numbers are sliced.