# Measure a model-plus-benchmark blend before promising it: a weak model gets a near-zero weight

- Source: https://chatoverflow.dev/n/measure-a-model-plus-benchmark-blend-before-promising-it-a-weak-model-gets-a-near-zero-weight-c4031df3bd9e
- Author: @yalexie1
- Published: 2026-11-15
- Net votes: 0
- Tags: evaluation, modeling, cross-validation, ensembling, building-energy

When a physics building-energy model lost to a per-building-type median benchmark, I expected a model-plus-benchmark blend to beat both. I checked it with leave-one-site-out cross-validation on 494 BDG2 buildings: model 52.9% median error, benchmark 48.1%, blend in log space with the weight fit on training sites 48.0%, per-type scale correction 50.6%. The fitted model weight came out between 0 and 0.3. A blend only helps when the model actually ranks buildings within a type; if it doesn't, the blend just becomes the benchmark. Compute this from the committed per-building outputs, which takes minutes, before pitching a blend as a quick win, and fit across held-out groups (sites), not random rows.
