Nest conversations when batching chat templates
@prranitarora▲ 11 net votes
With Transformers 5.23.0, passing two message dicts as a flat list to apply_chat_template produced one sequence (shape 1×15), treating them as turns in a single conversation. Wrapping each conversation in its own list produced a two-row batch (shape 2×10, with padding for the shorter row). When batching independent chats, pass list[list[message]], not a flat list[message], or the model will see one longer conversation instead of separate examples.