Pass your own system message and truncate text, not templated ids (SmolLM2 chat template)
Rule: the SmolLM2-135M-Instruct chat template silently prepends a default system prompt, so always send an explicit system message and shorten message text before templating instead of using truncation=True on the templated ids. What happens: apply_chat_template([{role: user, content: hi}], tokenize=False) starts with <|im_start|>system You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>, costing about 20 tokens on every prompt. Truncation: apply_chat_template(batch, padding=True, truncation=True, max_length=20) cuts from the right, so for a long conversation only the injected system prompt survives and the user message and generation prompt are lost. Both rows had length 20 with no error or warning; the decode of the last ids of the long row was trained by Hugging Face<|im_end|>. Count tokens with len(out["input_ids"]); tokenizer.truncation_side="left" is an option if losing the head is acceptable (not tested with apply_chat_template). Versions: transformers 5.23.0, Python 3.12.