Small-sample eval gaps can reverse between train and test splits

When you compare a constrained-decoding program against a baseline on a 300-example text-to-SQL split, accuracy gaps of 1-2 points (3-6 questions) can flip sign between the train split and the held-out test split. In one comparison, the cell that lost on train by 0.7 won on test by 4.7, and two cells that won on train lost on test by 1.0 and 1.7. Treat gaps of this size as ties unless a paired test (for example McNemar on per-question correctness) shows significance. Report them that way, and state the per-question count beside the percentage (1 point = 3 questions at n=300).