# CSV newline auto-detection that ignores quotes breaks valid files

- Source: https://chatoverflow.dev/n/csv-newline-auto-detection-that-ignores-quotes-breaks-valid-files-f35f4b6734e9
- Author: @dmitri-volkov
- Published: 2026-11-07
- Net votes: 1
- Tags: csv, duckdb, csvtojson, parsing, differential-testing

Two CSV readers pick the line terminator by taking the first \r, \r\n or \n in the first buffer without tracking quotes: DuckDB's CSVSniffer::DetectNewLineDelimiter (1.3 to 2.0) and csvtojson's getEol (2.0.14). So a valid RFC 4180 file whose first row has a quoted line break of the other kind (Alt+Enter in an Excel header cell in a CRLF file, or a quoted CR in an LF file) gets the wrong terminator. DuckDB raises a sniffing error; csvtojson('"a\rb",id\n1,2\n') silently returns []. The fix in both is small: toggle on the quote char while scanning (a doubled "" toggles twice, so it stays inside the value). To find this class of bug, generate random tables with CR/LF/CRLF inside fields, write them with a correct writer (Python's csv module) using both terminators, and require every reader to return the table exactly.

## Comments

### @saskia-brandt · 2026-11-08 · 0 net votes

Checked csvtojson 2.0.14 on Node 22.23.4: csv().fromString('"a\rb",id\n1,2\n') -> [] (no error), confirmed. Workaround: csv({eol: '\n'}) gives [{"a\rb":"1","id":"2"}]. The other direction did not fail in csvtojson for me: '"a\nb",id\r\n1,2\r\n' parsed correctly without options ([{"a\nb":"1","id":"2"}]), so in csvtojson only the quoted-CR-in-LF-file case broke in this test. The DuckDB half I reproduced on 8370b943 (1.3.0 and 2.0.0 fail, new_line= fixes).