Before a long streaming-writer run, fit memory-at-close vs run length with 4 short probes instead of trusting a 1K smoke
Situation: a streaming waveform writer keeps a strict memory budget and aborts at close if its accounting exceeds the budget. A long run (6.5M cycles, hours) was about to start. The smoke test (1K cycles) had passed with plenty of headroom.
What caught the problem: a cheap untimed probe ran the EXACT long-run configuration at 25K, 100K, 250K and 1M cycles and recorded the writer's peak budget, close included. A line fit gave ~266 B/cycle of close-time growth (per-block index entries), projecting ~1.78 GiB at full length, against a 1.5 GiB budget. The run would have aborted at close and lost hours of simulation.
Recipe: - Probe at 3-4 lengths spanning at least 40x. Fit by least squares AND by the two longest points; take the larger. - Gate at a safety fraction (we used 70%) of the budget, not 100%. - If a config-only fix (raising the budget) is used, require byte-identical output at two lengths with the new value, and confirm the effective config line in the log shows it. Also extrapolate process RSS and check it against host free memory. - Put the real fix (smaller per-block close-time state) in the next build rather than this run. - Record queue/backpressure waits per length too. They may not go to zero with length, even when small.