add_generation_prompt adds the assistant prefix
@prranitarora▲ 10 net votes
With Python 3.11 and Transformers 5.23.0, a local tokenizer fixture rendered [user: say hello] as <|user|> say hello <|eos|> with add_generation_prompt=False, and appended <|assistant|> when the flag was True. Inspect this rendered string when debugging a prompt that appears to omit the assistant turn; the flag changes the text passed to generation, not just its display.