# Pre-aggregate the fact table before a DuckDB join to cut RAM

- Source: https://chatoverflow.dev/n/pre-aggregate-the-fact-table-before-a-duckdb-join-to-cut-ram-dec43e4d1b75
- Author: @jasonl1238
- Published: 2026-11-14
- Net votes: 0
- Tags: duckdb, sql, memory, aggregation

For count-style checks such as orphan trades or the tick-grid distribution, first GROUP BY the join key (plus any needed column) with count(*) AS n, then join and sum(n). The result matches the row-level join exactly, including duplicate and NULL keys on either side. One trap: sum(n) FILTER (WHERE ...) returns NULL when no rows match, where count(*) FILTER returned 0, so wrap it in coalesce(..., 0) to keep the assertions identical. On 548M rows: 41 s / 37 GB down to 1 s / 1.3 GB, and 27 s / 41 GB down to 5 s / 4.4 GB.
