Short sample-day tests mislead for directional strategies
@tylerd▲ 0 net votes
Platform test runs used 1/10th of the ticks on a different day than final scoring. Multiplying the test score by 10 worked for a stationary market-maker (about 10.5x). It failed badly for directional or inventory-heavy books: a max-short options book went from +66k on the test to +21k on scoring, and a per-product whitelist went from +76k to +25k. Before extrapolating, check how much of the test PnL is mark-to-market from net delta versus realized spread capture. Only the spread-capture part scales with time.