One-column CSV with NULLs: pandas drops rows that polars and DuckDB keep
polars write_csv and DuckDB COPY (default null string '') write a NULL in a one-column table as an empty line. polars 2.1 and duckdb 2.0 read that line back as a NULL row, but pandas 3.1 read_csv (skip_blank_lines=True by default) drops it, so a 3-row file written by polars arrives in pandas with 2 rows. In the other direction, duckdb's allow_quoted_nulls=true default turns pandas' "" (empty string) into NULL. For one-column exchange files, write an explicit null marker (polars null_value='NULL', DuckDB NULLSTR 'NULL' together with FORCE_QUOTE, pandas na_rep), and read with pandas skip_blank_lines=False / DuckDB allow_quoted_nulls=false. Or use Parquet.
Reproduced with polars 2.1.0, pandas 3.1.1, duckdb 2.0.0: polars and DuckDB both write b'v\na\n\nb\n'; pandas reads ['a', 'b'] (skip_blank_lines=False gives ['a', nan, 'b']); polars and duckdb read 3 rows. One addition about the pandas side: in a one-column frame pandas to_csv writes NaN as "" too, the same as '' (['a', '', None, 'b'] -> b'v\na\n""\n""\nb\n'). So allow_quoted_nulls=false turns pandas' NaN into '' as well ([a, '', '', b]), and polars also reads '' twice. pandas reading its own file gives [a, nan, nan, b]. na_rep='NULL' fixes the writer (b'v\na\n""\nNULL\nb\n'); duckdb (nullstr='NULL', allow_quoted_nulls=false) and polars (null_values='NULL') then return a, '', NULL, b exactly. pandas itself still needs keep_default_na=False, na_values=['NULL'] to read the '' back.