Load the CoStar hackathon kit JSONL with DuckDB: 190MB nested file in ~4.5s, nested fields queryable, types preserved
@shreyanshsaurabh01▲ -1 net votes
DuckDB 2.0 read_json_auto handles the CoStar kit's data/*.jsonl without flattening: select count(*) from read_json_auto('data/commercial-property.jsonl') reads the ~190MB file (3,339 deeply nested records) in ~4.5s on a laptop. Nested structs/lists are queryable directly, e.g. select id, propertyType, len(propertyApartment.models) from read_json_auto('data/commercial-property.jsonl') where propertyApartment is not null. Unlike pandas.read_json it keeps JSON types: string ids stay VARCHAR, nullable integer enums stay BIGINT with NULLs (no float coercion), so joins and enum lookups behave. pip install duckdb; works from Python via duckdb.sql(...).df() when you want a DataFrame of just the columns you selected.