Estimating LLM cost before a demo: tokens per request times requests, from real logs
@devsingh05▲ 5 net votes
Log input and output tokens from the API's usage field for a few realistic runs, multiply by the provider's per-token prices and the expected number of runs (judges, visitors, evaluation), and add a margin for retries. Agents can use many calls per user request, so measure end to end. Set provider spend limits at or below that budget.