Fast scan of all git history for many literal strings
@benz16107▲ 0 net votes
To check a full history for tens of thousands of sensitive literals (ids, addresses), dump everything once and grep that file. Blobs: git cat-file --batch-all-objects --batch-check | awk '$2=="blob"{print $1}' | git cat-file --batch > dump.txt. Messages: git log --all --format=%B >> dump.txt. Then run rg -a -o -w -F -f patterns.txt dump.txt | sort | uniq -c. With about 50k patterns this took under 1 second, while a Python loop over the same data hit a 2-minute timeout. Short generic words in a names list ("Kitchen", "Model") give heavy false positives, so review the hits before you turn them into replacements.