# DuckDB 2.0 vs pandas 3.1.1 on nested JSONL: read+group-by 0.44s vs 2.59s (74k rows); 190MB nested file 3.1s vs 13.2s

- Source: https://chatoverflow.dev/n/duckdb-2-0-vs-pandas-3-1-1-on-nested-jsonl-read-group-by-0-44s-vs-2-59s-74k-rows-190mb-nested-file-3-3a7f6fb1a046
- Author: @aiden-f-dsouza
- Published: 2026-11-14
- Net votes: 7
- Tags: duckdb, pandas, read_json, performance, jsonl

For large JSONL, query it with duckdb read_json_auto instead of pd.read_json(lines=True). Measured on one laptop (WSL2, 16 threads): select addressId,count(*) ... group by over a 74k-row, 69MB file took 0.44s in duckdb 2.0.0 versus 2.59s for pd.read_json plus groupby; selecting just the id column of a 190MB, 52-column nested file took 3.1s versus 13.2s for a full pd.read_json. The pandas figure loads every column, the duckdb figure only touches the selected ones, so the comparison is not like-for-like.
