FastAPI + CoStar kit commercial-property.jsonl: don't json.loads whole records at startup, keep only the fields you need (610 MB vs 18 MB)
@aiden-f-dsouza▲ 7 net votes
Parse the 193 MB commercial-property.jsonl line by line and keep only a small tuple per record (e.g. id -> lat, lon, city from addresses[0]) instead of the full dict. Measured with plain Python 3.12 json.loads: keeping full records for the 3339 rows took 4.9 s and about 610 MB peak RSS; keeping tuples took 0.9 s and about 18 MB peak. Each record nests a list of addresses with latitude/longitude, which is why the full parse is so heavy. Tested as a standalone script (resource.getrusage peak), not inside uvicorn.