Seeding a synthetic baseline for a median/MAD anomaly check: sample density per comparison window decides the false-alarm rate
Situation: a detector scores each new reading against the series' own history using a robust z (median, MAD scaled by 1.4826) plus a two-metric Mahalanobis distance, and it only compares readings from the same time of day (a +/-3 hour window). A new series has no history, so a week of plausible estimated history was generated to stand in.
What went wrong first: with one synthetic point every 3 hours (56 points), each time-of-day window held only about 16 points. MAD and the sample covariance are noisy at that size, and about 5% of perfectly ordinary new readings were flagged as unusual. Every 2 hours was worse (about 10%), because a slow-shift chart (EWMA/CUSUM) started firing as well. One point per hour (168 points, about 40 per window) dropped it to under 1%, with clear outliers still caught every time.
What helped: - Do not tune by eye. Write a 30-line sweep: for many seeds and every hour of the day, build the synthetic history, score a handful of ordinary readings and a handful of obvious outliers, and count verdicts. It took seconds and overturned the first two guesses. - Make the synthetic scatter wider than you think (here about 18% of the centre) when the true level is unknown; a tight baseline turns normal readings into alarms. - Also score the synthetic series against itself (last point vs the rest). If it flags itself, the generator and the detector disagree. - Clamp generated values inside the detector's hard thresholds so no seeded row carries a contradictory label. - Tag seeded rows (a reserved session or source id) so they can be counted, excluded from exports and deleted in one statement, and seed only once per database.