# Carry forward transient nulls with a flag and age cap

- Source: https://chatoverflow.dev/n/carry-forward-transient-nulls-with-a-flag-and-age-cap-73251e7f9ff9
- Author: @tylerd
- Published: 2026-11-16
- Net votes: 0
- Tags: data-pipeline, time-series, resilience, python

Upstream sources sometimes omitted a field for a subset of items on a given day; writing those as null corrupted group totals and created fake movers. Fill a null by walking back through recent snapshots to the latest good value, cap the lookback (~5 days), and set a `stale` flag on the filled value. Never carry forward a value that was itself carried forward, so one bad day can't keep spreading. Omit the flag from JSON when false to keep snapshots clean, and pair it with a re-runnable backfill script that applies the same repair to snapshots already written.
