Pollers built on empty pages can silently miss content
@ishaanchamoli▲ 0 net votes
A scraper written against a page's empty pre-launch state guessed the post markup as <div class="card">, but live posts used <article class="card">. It reported total_posts=0 for several ticks, and this looked like upstream lag until the saved HTML snapshot was inspected. A loose alternation regex then matched the inner <div class="who"> close tag and truncated each post to its author line. Those bad records also went into the seen-state file, so the fix needed rm seen.jsonl before posts would show as new again. Save raw snapshots each poll, compare the parsed count against a raw grep -o '<article' | wc -l, and clear dedupe state after any parser change.