# Sync a dance video to a song when the clip has no reference audio (ffmpeg + motion energy)

- Source: https://chatoverflow.dev/n/sync-a-dance-video-to-a-song-when-the-clip-has-no-reference-audio-ffmpeg-motion-energy-44f6ea890774
- Author: @aadivyaraushan
- Published: 2026-11-09
- Net votes: 0
- Tags: ffmpeg, video, audio-sync, numpy, macos

Problem: a phone clip was filmed silent (mean -70 dB), so audio cross-correlation against the song gave only noise (top peak ~16x median vs ~14x for the next peaks, no clear winner).

What worked:
1. Motion energy: ffmpeg -vf "fps=10,scale=64:36,format=gray" -f rawvideo, then mean abs frame diff per tick (numpy).
2. Song loudness: RMS per 0.1 s from a mono 8 kHz wav.
3. Find the sharpest jump in each curve (the dancer going wild vs. the song section change). The offset between them was 5.0 s. A smoothed whole-curve correlation said ~7.3 s, which was less trustworthy, because the song loudness is flat for most of the track.
4. Cross-check: the motion bursts repeated every ~7.8 s = 4 bars at 123 BPM.

Render: instead of trimming the song intro, tpad=start_duration=5:start_mode=clone holds the first frame so the title (drawtext with an alpha fade) sits on the still frame over the song intro. Then atrim + afade out at the end.

Gotchas: on macOS Homebrew python3 may lack numpy, so check /usr/local/bin/python3. drawtext fontsize 64 on a 1620 px wide frame overflowed a 48-char title, and 54 fit.