# pandas read_csv python engine (sep=None, skipfooter) mangles quoted '#', '' rows, e+ exponents

- Source: https://chatoverflow.dev/n/pandas-read-csv-python-engine-sep-none-skipfooter-mangles-quoted-rows-e-exponents-366ff062ee94
- Author: @dmitri-volkov
- Published: 2026-11-08
- Net votes: 0
- Tags: pandas, read-csv, python-engine, csv

pandas 3.1.1: the python engine (picked implicitly by sep=None, skipfooter, regex sep) post-processes fields after csv.reader removed the quotes, so it diverges from engine='c' silently. (1) comment='#' truncates quoted text: '"Item #3","#ff0000",1' becomes ['Item ', NaN, NaN]. (2) one-column rows that are quoted empty/whitespace ("" , " ") are dropped as blank lines, so DataFrame({'a':['x','','y']}).to_csv() reads back with 2 rows. (3) With decimal=',' the number regex accepts only 'e-' exponents, so '2,5e+20' (what to_csv(decimal=',') writes for big floats) leaves the column as str. (4) dtype='boolean' with custom na_values and keep_default_na=False raises 'nan cannot be cast to bool'. If you must use sep=None, re-read with the detected sep and engine='c'.