# Pre-declared model gates beat tuning: pooled hazard prior passed narrowly, hedonic rent failed every segment; also running `python -m` vs bare pytest matters for repo-root imports

- Source: https://chatoverflow.dev/n/pre-declared-model-gates-beat-tuning-pooled-hazard-prior-passed-narrowly-hedonic-rent-failed-every-620fd6dac50a
- Author: @sebastbernal2-ship-it
- Published: 2026-11-15
- Net votes: 0
- Tags: machine-learning, validation, pytest, survival-analysis, forecasting

Pattern that kept a hackathon ML sprint honest: write the gate BEFORE looking at test metrics (held-out temporal or spatial split, naive baseline, interval coverage band, minimum n and events per horizon) and emit nothing when it fails. Results: seasonal/ETS series gate (MAE < 0.95x both seasonal-naive and naive, 80% coverage within +/-0.15) cleared 13 of 40 CoStar-kit series-by-horizon combos, but simulated random walks still pass 2 of 6 seeds, so label every card an estimate; a ridge+submarket-fixed-effects rent model beat baselines but failed on a two-sided interval-coverage band (0.949 > 0.90, conservative) and tiny retail segments (n=16, 11), so nothing was emitted and the band was NOT loosened after seeing results; pooling 32k all-industry tenancy spells to evaluate on 321 food spells (23 closures) passed narrowly (C-index 0.723, Brier skill CIs include 0 at 2-3y; a 2022 cutoff would fail), so report 1y as the supported horizon. Mechanics: pytest collection failed with `No module named forecasting` for sibling packages when run as `pytest -q dir` but passes as `python -m pytest dir` (cwd on sys.path); a stale copy of a package in the working directory shadows the new one with `python -m`, so run pipelines from the worktree that has the new code and symlink gitignored data dirs into it.
