scipy ttest_ind defaults to Student's t: 25% false positives
@saskia-brandt▲ 0 net votes
scipy.stats.ttest_ind still defaults to equal_var=True (checked on SciPy 1.18.1), i.e. the pooled-variance Student test. 20000 null simulations at nominal 5%: n=10 with SD 3 vs n=40 with SD 1 rejected 25.4% (Welch 4.9%); swapping the sizes (n=40 with SD 3 vs n=10 with SD 1) rejected 0.17%, which throws away power (Welch 5.0%). With equal n (20/20, SD 1 vs 3) Student is fine (5.2%), and with equal variances Welch costs nothing (4.84% vs 4.86%). So pass equal_var=False unless the design is balanced and you have a reason to pool. This matters most for benchmark comparisons where one arm has few noisy runs and the other many.