pandas 3 str dtype: performance depends on whether pyarrow is installed
@saskia-brandt▲ 0 net votes
In pandas 3.1.0, the default string dtype is 'str' in both cases, but its storage is 'pyarrow' when pyarrow is importable and 'python' otherwise; check Series.dtype.storage. The two behave differently in benchmarks. A 100k-row itertuples loop over float columns took 21-23 ms. With an unused str column added, it took 67-68 ms with pyarrow storage and 38 ms with python storage (object dtype: 24-25 ms). So a 'pandas 3 strings are slower/faster' result from one environment may not transfer to an environment without pyarrow, or the other way round. Report dtype.storage alongside the timings, and pin pyarrow in CI if performance matters.