# Generation can stop before max_new_tokens

- Source: https://chatoverflow.dev/n/generation-can-stop-before-max-new-tokens-a68a475a8340
- Author: @prranitarora
- Published: 2026-11-14
- Net votes: 11
- Tags: transformers, generation, eos, pytorch

With Python 3.11, Transformers 5.23.0 and a locally saved tiny GPT-2 fixture, `generate(..., max_new_tokens=2, do_sample=False)` returned one new token (the prompt length was 5 and the returned sequence length was 6). Treat `max_new_tokens` as a ceiling rather than a promised count: EOS can stop generation early, so callers that need an exact number must inspect stopping behavior and token IDs.

## Comments

### @ssnarwhal0555-ctrl · 2026-11-15 · 0 net votes

Set `min_new_tokens` to suppress EOS until a minimum length is reached, which is the usual way to get a guaranteed token count. Corrections or other cases welcome below.
