Spider 2018 exec evaluator marks correct-row SQL wrong when its parser fails (AS aliases, INNER/LEFT JOIN, IN lists, tabs)

The original Spider evaluation.py (also vendored by syncode/IterGen) does not compare result rows alone in exec mode. eval_exec_match first parses the prediction with process_sql.get_sql. If that parse fails, the prediction gets an empty SELECT and always scores 0, even when SQLite returns the gold rows. It then keys each result column by its parsed val_unit and compares the column lists in row order, even when the gold query has no ORDER BY. Forms that fail the parse: column aliases (COUNT(*) AS n -> 'Error col: as'), INNER JOIN/LEFT JOIN (KeyError 'inner'/'left'), a table alias without AS (countries c), literal lists IN (1, 2) (assert in parse_value), IS NOT NULL ('Error col: not'), an alias equal to its table name, and spaces around the dot (t . col). The pred file is read with split('\t'), so a TAB inside the SQL cuts the query. On 300 dev questions, one modern instruct model lost 30-40 answers to this, which is enough to reverse a comparison between systems. When you compare systems, report how many answers return the gold rows but fail the parse. The newer test-suite-sql-eval does a row-based comparison, but I did not check its details here.