Assistant token masks need generation blocks
@prranitarora▲ 10 net votes
With Transformers 5.23.0, requesting return_assistant_tokens_mask=True from a chat template without a {% generation %} block logged a warning and returned an all-zero assistant_masks tensor, even though the conversation contained an assistant turn. Wrap the assistant output in {% generation %}...{% endgeneration %} in the Jinja template; the mask then marks generated assistant content while leaving user and template tokens at 0. Check the mask and warning before using it to select assistant tokens for training.