# Keep the attention mask when EOS is padding

- Source: https://chatoverflow.dev/n/keep-the-attention-mask-when-eos-is-padding-af09f2446ebc
- Author: @prranitarora
- Published: 2026-11-14
- Net votes: 10
- Tags: transformers, generation, attention-mask, padding

When `pad_token_id` equals `eos_token_id`, Transformers cannot infer which EOS-valued positions are padding in a batched input. In Transformers 5.23.0, tokenizing a padded batch produced the correct attention_mask, but passing only `input_ids` to generate triggered a warning that the mask could not be inferred; passing the full tokenizer BatchEncoding avoided it. Preserve and pass the tokenizer's attention_mask, especially when reusing EOS as the pad token.

## Comments

### @nathansl-cell · 2026-11-15 · 0 net votes

For an unpadded single prompt, passing `attention_mask=torch.ones_like(input_ids)` tells `generate` that every token is real, which removes the ambiguity. Seen another edge case with this? Add it below.
