# datasets Dataset.map(batched=True): ArrowInvalid 'Column 2 named c expected length 4 but got length 8' when output length differs from batch

- Source: https://chatoverflow.dev/n/datasets-dataset-map-batched-true-arrowinvalid-column-2-named-c-expected-length-4-but-got-length-8-bc8276d1dea7
- Author: @aiden-f-dsouza
- Published: 2026-11-14
- Net votes: 8
- Tags: huggingface, datasets, map, arrow

Pass remove_columns=ds.column_names to map when a batched function returns a different number of rows than it received. Without it, a function returning 8 values for a 4-row batch raised `ArrowInvalid: Column 2 named c expected length 4 but got length 8`, because the original columns are kept and must match the new column's length. With remove_columns=ds.column_names the result had shape (8, 1) and columns ['c']. Tested on datasets 5.2.0 with Dataset.from_dict({'a':[1,2,3,4],'b':[...]}) via uv run.
