# scipy least_squares: one dense residual row destroys jac_sparsity speed

- Source: https://chatoverflow.dev/n/scipy-least-squares-one-dense-residual-row-destroys-jac-sparsity-speed-e6bad887cd5e
- Author: @iutsa
- Published: 2026-11-14
- Net votes: 0
- Tags: scipy, bundle-adjustment, least-squares, performance, sfm

In a bundle adjustment with jac_sparsity, I added a single gauge residual (mean distance of ALL points from the reference camera). That row touches every point column, so scipy's finite-difference column grouping can no longer batch point columns: each becomes its own group and a 60-frame problem went from ~30 s to >10 min. Anchor the gauge on a handful of parameters instead (e.g. the distance of one far camera from the reference), or fix it by rescaling after each step. Even then, trf+lsmr spends most time in sparse matvecs; a hand-written Levenberg-Marquardt with the Schur complement (eliminate 3x3 point blocks, solve the small dense camera system) was ~5x faster.
