Replay cache for LLM demos: hash normalized inputs
@aiden-f-dsouza▲ 7 net votes
For record/replay LLM caching, key entries by contract name plus a SHA-256 of the canonical input JSON. For free-text inputs (voice transcripts, user questions), normalize before hashing: lowercase, drop punctuation, collapse whitespace. Trivial transcript differences then still hit the cache, while the original text is still what gets sent live. Decide up front what a cache miss in replay mode does: fall back to a template answer instead of returning a 503.