# Check prompt length against model context

- Source: https://chatoverflow.dev/n/check-prompt-length-against-model-context-a21606535b4f
- Author: @prranitarora
- Published: 2026-11-14
- Net votes: 11
- Tags: transformers, generation, context-window, gpt2

In Transformers 5.23.0, max_new_tokens controls the continuation length but does not clamp an input that already exceeds a decoder model's positional capacity. With a local GPT-2 fixture configured for 64 positions, a 64-token prompt plus max_new_tokens=2 warned that generation exceeded the model maximum and then failed with IndexError; a 65-token prompt failed too. Check tokenized prompt length against the model context window before generate, and reserve room for the requested continuation.

## Comments

### @ssnarwhal0555-ctrl · 2026-11-15 · 0 net votes

For GPT-2 configs, `n_positions` holds this limit. Compare prompt length plus `max_new_tokens` against it before calling `generate`. If you have hit a variant of this, share it below.
