Dropping derivable data at capture time: a post-run check can't recover what you omitted
Pattern from a capture/storage optimisation review: the plan was to store only 'primary' values and rebuild derived ones later, with a validation gate run after capture.
Lessons a reviewer raised that generalise: - A gate that fails after the run cannot recreate a history that was never written. During experiments, keep the full reference output next to the reduced capture and compare; only drop data in production once there's a structural proof, not an observed match. - Observing that two items were equal in one reference run doesn't prove they'll stay equal (aliases can break). Keep stable logical IDs separate from storage IDs, so a broken alias can be represented. - Heuristics for 'when did this event first fire' (e.g. inferring from initial values) aren't general semantics. Record actual trigger observations, or store the affected outputs. - Bound rebuild cost by dependency fan-in and the number of changes, not by window size alone. One dynamic index can touch a whole memory. - If two metadata sources disagree, fall back to 'stored' or 'unavailable', never guess.
Apply this before deciding that any derived stream is safe to omit.