Recording an X11-forwarded GUI into a true-time video: xwd on Xvfb, convert per frame, concat durations

Goal: a video of a remote GUI (ssh -X) shown on a private local Xvfb (XQuartz /opt/X11/bin/Xvfb :N, nothing on screen), with real timing so lag is visible.

What worked: - Capture with xwd -root -display :N -silent in a loop and record time.time() per frame. At 1440x900 each raw XWD is ~5 MB, so 0.1 s capture over a few minutes is many GB. Convert each frame to PNG in a worker thread as soon as it is written and delete the XWD, or the disk fills. - Write the frame timestamps to a file BEFORE any conversion step. A capture taken while the remote app exits can be 0 bytes or truncated; ffmpeg -i bad.xwd then fails ('Invalid data found'), and with check=True the script died and lost every timestamp. Skip empty or failed frames instead. - Build the video with the ffmpeg concat demuxer using real gaps: lines file 'NNNNN.png' + duration <next_t - t>, repeat the last file once, then -vf scale=1200:-2,format=yuv420p -r 10 -c:v libx264 -movflags +faststart. - To place remote-side event marks (e.g. a Tcl script appending name [clock milliseconds] to a log) on the local time axis, estimate the clock offset from a few ssh host date +%s%3N round trips and keep the midpoint of the fastest one. - A command 'returning' in the remote script is not when the screen changed. Find the first frame after the send that differs from the previous one (downscale to gray with ffmpeg -f rawvideo and count changed pixels) and the last change before the next step: that is the user-visible latency. - Pace scripted interactions with after 2000 next style scheduling rather than back-to-back calls, so redraws are not coalesced and each step is visible.