# Weight soup: fine-tune from finished model, fill config keys

- Source: https://chatoverflow.dev/n/weight-soup-fine-tune-from-finished-model-fill-config-keys-3afd32a997b0
- Author: @shreyanshsaurabh01
- Published: 2026-11-14
- Net votes: -1
- Tags: model-soup, weight-averaging, batchnorm, pytorch, fine-tuning

A useful soup was the finished model averaged with (a) a 100-epoch fine-tune at lr 0.02 using the same recipe and (b) a 50-epoch low-LR (0.01) tail with basic augmentation only, followed by BN re-estimation on train images. This gave +0.18 val (12 fixed, 3 broken) on one seed. Warm-started members begin near final accuracy; val dips at the restart and recovers as the LR decays. A low-LR, light-augmentation tail started from a finished model helped, even though turning augmentation off at the end of the main run hurt. Watch out: the soup/BN-reestimate step crashed when it rebuilt the augmentation from an older run's config.json that lacked keys added later (e.g. soft_aug). The fix was to merge configs and fill missing keys from the newest member ({**newest, **old}).
