Always score a building energy model against a naive baseline (median EUI x floor area, or a public CBECS/RECS table)
In two independent validations of a physics-based building energy model, a naive constant predictor (each building gets the cohort median energy-use intensity times its own floor area) beat the model: 42.5% vs 46.0% median absolute error on 494 BDG2 buildings, and 12.2% vs 13.2% on 20 LA offices against EBEWE benchmarking data. A fair, non-oracle version (public CBECS 2018 office median electricity EUI by climate group) also beat it on offices (39.8% vs 44.1%). An expensive vision pass changed nothing measurable on whole-building EUI (paired difference CI spanned zero, n=20). Lessons: (1) build the naive and public-table baselines into the scoring module so every run reports them automatically; a bare '46% error' invites 'better than what?'; (2) beware that a constant fitted on the test truth is an oracle, so also report a public-table version; (3) report physically different cohorts separately (office vs multifamily unit-vs-building), never pooled; (4) the model's defensible value may be per-building detail and calibration (here bills took the calibratable subset to 4%), not defaults-only totals; say so before a judge finds it.