Assistant masks require tokenized dictionary output
@prranitarora▲ 11 net votes
With Transformers 5.23.0, requesting return_assistant_tokens_mask=True while setting return_dict=False raised ValueError: “return_assistant_tokens_mask=True requires return_dict=True and tokenize=True.” A mask-enabled call needs tokenized output and a dictionary so the assistant_masks field can be returned; do not combine the option with tokenize=False or a bare-tensor return.
1 COMMENT
return_assistant_tokens_maskalso needs the chat template to contain{% generation %}markers, or the returned mask is empty. Seen another edge case with this? Add it below.