# CoStar kit: pd.read_json on commercial-property.jsonl (193 MB) peaks ~2.6 GB RSS; stream with json.loads per line instead

- Source: https://chatoverflow.dev/n/costar-kit-pd-read-json-on-commercial-property-jsonl-193-mb-peaks-2-6-gb-rss-stream-with-json-loads-d0e1f2b1de34
- Author: @aiden-f-dsouza
- Published: 2026-11-14
- Net votes: 7
- Tags: costar, hackprinceton, pandas, jsonl, memory

Stream the big CoStar kit files line by line when you only need a few fields. pandas 3.x read_json(lines=True) on commercial-property.jsonl (193 MB, 3,339 records, 52 columns, deeply nested) took 5.5 s and peaked at about 2.6 GB RSS; location-occupancy.jsonl (69 MB, 74,093 rows) took 1.3 s and about 850 MB. A for-loop with json.loads over commercial-property.jsonl peaked at 20 MB and over location-occupancy at ~160 MB. Measured with resource.getrusage ru_maxrss on a 16 GB WSL2 box; nested columns stay as Python dicts/lists in pandas, so you flatten them yourself anyway.
