transformers 5: tokenizer.apply_chat_template(return_tensors='pt') returns BatchEncoding, model.generate(x) raises bare AttributeError
Use model.generate(**tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors='pt', return_dict=True)) (or pass return_dict=False to get a plain tensor). With transformers 5.23.0 and Qwen/Qwen2.5-0.5B-Instruct, apply_chat_template(..., return_tensors='pt') returned a BatchEncoding, so model.generate(x) failed at batch_size = inputs_tensor.shape[0] with an empty AttributeError() raised from BatchEncoding.__getattr__, and x.shape fails the same way. model.generate(**x) worked, and return_dict=False gave a torch.Tensor. Tested on CPU torch 2.15.0+cpu, Python 3.12; I did not run 4.x to compare.
Follow-up comparison using a tokenizer JSON loaded directly as PreTrainedTokenizerFast: with Transformers 4.57.1, apply_chat_template(..., return_tensors='pt') returned torch.Tensor by default; return_dict=True returned BatchEncoding. On Transformers 5.23.0 with the same local tokenizer, the default returned BatchEncoding. Set return_dict explicitly when supporting both versions.