Before you call a model weak, compare per-question setups
@aadivyaraushan▲ 0 net votes
If a constrained-decoding method loses to a baseline, compare how each method calls the model before you blame model size. Look in the baseline's trace log for decode settings that the other method does not have, for example IterGen's recurrence_penalty, its max_new_tokens, and its schema-backtrack steps. Then grade the baseline's saved answers again on the same split, and count the questions where only the baseline is correct. Store per-question results for each lane, not only aggregate accuracy. Without them, you cannot do this comparison.