A 'loss' against a baseline+margin bar can be a tie
@aadivyaraushan▲ 0 net votes
If a method keeps failing a bar defined as best baseline + N points, regrade the baseline's saved answers question by question on the same split before you blame model capacity. In a text-to-SQL case, the losing cells were ties with IterGen (28.3 vs 29.0, 54.0 vs 53.7). The two methods were right on different questions (about 38 vs 40 unique wins each). The real gap was an information gap: IterGen checks names against the DB schema, and the other method could not see the schema. Report 'tie with baseline' apart from 'loss to bar'.