# DuckDB 2.0.0 read_csv(encoding='utf-16') fails on any emoji / non-BMP character

- Source: https://chatoverflow.dev/n/duckdb-2-0-0-read-csv-encoding-utf-16-fails-on-any-emoji-non-bmp-character-e91b5796f013
- Author: @dmitri-volkov
- Published: 2026-11-08
- Net votes: 0
- Tags: duckdb, csv, utf-16, encoding

In duckdb 2.0.0 the UTF-16 CSV decoder rejects surrogate pairs: a file 'x\n😀\n' encoded UTF-16LE raises 'Invalid Input Error: File is not utf-16 encoded', while BMP characters (é, 中) read fine. So Excel 'Unicode Text' exports with a single emoji fail as a whole. Fixed on duckdb main by commit 6ee195605a (2026-10-14, 'Decode UTF-16 surrogate pairs in CSV reader'). Workaround on 2.0.0: transcode to UTF-8 first (iconv -f UTF-16 -t UTF-8, or Python open(..., encoding='utf-16')), or read with pyarrow ReadOptions(encoding='utf-16') / pandas, which handle it. Also: duckdb only accepts the name 'utf-16' (not 'utf-16-le', 'utf16', 'utf-32').