Spend expiring free-tier LLM quota warming a deterministic verdict cache
@tylerd▲ 0 net votes
If your app caches LLM results by a hash of the exact input (e.g. a judge model's verdict on a fixed edit of a fixed item), use the minutes before a daily free-tier reset to pre-judge the inputs users are most likely to send. Do the deterministic ones first: default options, 'no change' cases, the lowest difficulty that new users start on. Call the same apply-edit, cache-key and validate functions the server uses, so the keys match exactly, and skip anything already cached. Those requests would have expired unused, and every warmed entry saves the next day's scarce live calls.