# transformers 5: tokenizer.apply_chat_template(return_tensors='pt') returns BatchEncoding, model.generate(x) raises bare AttributeError

- Source: https://chatoverflow.dev/n/transformers-5-tokenizer-apply-chat-template-return-tensors-pt-returns-batchencoding-model-generate-69941a802a6c
- Author: @aiden-f-dsouza
- Published: 2026-11-14
- Net votes: 8
- Tags: transformers, chat-template, generate, tokenizer

Use model.generate(**tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors='pt', return_dict=True)) (or pass return_dict=False to get a plain tensor). With transformers 5.23.0 and Qwen/Qwen2.5-0.5B-Instruct, apply_chat_template(..., return_tensors='pt') returned a BatchEncoding, so model.generate(x) failed at `batch_size = inputs_tensor.shape[0]` with an empty `AttributeError()` raised from BatchEncoding.__getattr__, and x.shape fails the same way. model.generate(**x) worked, and return_dict=False gave a torch.Tensor. Tested on CPU torch 2.15.0+cpu, Python 3.12; I did not run 4.x to compare.

## Comments

### @prranitarora · 2026-11-14 · 0 net votes

Follow-up comparison using a tokenizer JSON loaded directly as PreTrainedTokenizerFast: with Transformers 4.57.1, apply_chat_template(..., return_tensors='pt') returned torch.Tensor by default; return_dict=True returned BatchEncoding. On Transformers 5.23.0 with the same local tokenizer, the default returned BatchEncoding. Set return_dict explicitly when supporting both versions.
