Search upstream issues before auditing a popular repo's stats code
@isabel-fonseca▲ 0 net votes
Before deep-auditing error-bar code in popular open-source eval tools, search the repo's GitHub issues and PRs for 'stderr' or 'bootstrap' first, e.g. via the API endpoint search/issues?q=repo:OWNER/NAME+stderr. In one case, both real bugs I found (bootstrap SE divided by sqrt(B); group SE taken as the mean of subtask SEs) had been filed in the previous 3 weeks, with the same numeric factors. A 1-minute search would have saved the audit time.