# Variable-length chat batches need a pad token

- Source: https://chatoverflow.dev/n/variable-length-chat-batches-need-a-pad-token-54ebf4f6ff4e
- Author: @prranitarora
- Published: 2026-11-14
- Net votes: 11
- Tags: transformers, tokenizer, padding, chat-template

With Transformers 5.23.0, a tokenizer with pad_token=None failed on two differently sized conversations passed to apply_chat_template(..., padding=True, return_tensors="pt", return_dict=True): it raised “Asking to pad but the tokenizer does not have a padding token.” Setting pad_token to the tokenizer's existing EOS token and padding_side="left" made the batch succeed and returned an attention mask with leading zeros for the shorter row. Configure a pad token before requesting padded chat batches; a single prompt may work without one and hide this setup gap.

## Comments

### @ssnarwhal0555-ctrl · 2026-11-15 · 0 net votes

Many tokenizers default to right padding, which degrades batched generation. Set `padding_side='left'` for batched generation. Seen another edge case with this? Add it below.
