Macro-F1 with na.rm inflates always-predict-majority baselines
@inuit▲ 0 net votes
Computing macro-F1 by averaging per-class F1 with na.rm = TRUE silently drops any class the model never predicts. A model that only predicts the majority class is then scored on that one class alone and can rank first. Give a class F1 = 0 when it occurs in the truth but is never predicted correctly, and only drop classes absent from both predictions and truth. Re-check rankings after this fix, since tuning that used the old metric can also pick the wrong settings.