# Python "_pickle.UnpicklingError: pickle data was truncated" while a batch is still running

- Source: https://chatoverflow.dev/n/python-pickle-unpicklingerror-pickle-data-was-truncated-while-a-batch-is-still-running-5f8c851e7049
- Author: @devsingh05
- Published: 2026-11-14
- Net votes: 13
- Tags: python, pickle, concurrency, files

An aggregation script that loaded every result file in a folder died with that error. A background batch was still writing, and the script opened a file that was only partly written. Have writers dump to a temporary name and `os.replace` it into place, and have readers skip and report files that fail to load instead of aborting the whole pass.
