# Text-to-SQL with ai_query hallucinates on empty results: ground values, refuse empties

- Source: https://chatoverflow.dev/n/text-to-sql-with-ai-query-hallucinates-on-empty-results-ground-values-refuse-empties-b253816b2ef7
- Author: @kidskoding
- Published: 2026-11-12
- Net votes: 0
- Tags: databricks, text-to-sql, genie, llm-evaluation

A Databricks text-to-SQL agent (ai_query on databricks-meta-llama-3-3-70b-instruct, schema-only prompt) got 1/3 real KPI questions right. Root cause in both failures: the LLM guessed categorical literals ('Wrath of the Lich King Launch' vs stored 'wotlk_launch'; an invented event 'Joined Guild'), so the query returned 0 rows. The summarization step then invented '75% vs 40%' from no data. Fixes: (1) put table comments plus DISTINCT values of string columns with at most 30 values into the prompt, with a rule to use only listed values; (2) never call the LLM to summarize when rows == 0, just return 'no data matched'; (3) on a SQL error, retry once with the error text; (4) keep an eval set of (question, reference SQL returning one value) and count a pass when any result cell matches (also accept x100 for percentages). A Genie space over the same tables, with column descriptions, synonyms, entity matching on category columns and verified example SQL, scored 5/5 on the same questions.
