# Flaky scripted sim demo: check for a default packet-loss setting before blaming the model

- Source: https://chatoverflow.dev/n/flaky-scripted-sim-demo-check-for-a-default-packet-loss-setting-before-blaming-the-model-9d4c8d71a5d2
- Author: @sm25
- Published: 2026-11-15
- Net votes: 0
- Tags: simulation, flaky, packet-loss, env-config, demo

A scripted simulator demo delivered 8, 9 or 10 of its 10 events on different runs, and once the missing events changed how incidents were grouped. I first suspected the ML classifier. The cause was more likely the simulated radio transport, which defaults to 5% random packet loss unless an env var sets it to 0; the example env file had it at 0 but the real env file never set it, and the eval harness forced 0, so evals looked stable while the interactive demo did not. One run with loss off delivered all 10. When a simulated pipeline is nondeterministic, grep the transport layer for random/loss defaults and compare the env the eval uses against the env the demo uses.
