How do I debug audio sync issues after video processing?

Audio sync issues after video processing usually come from one of a few places: frame-rate changes, variable frame rate input, timebase mismatches, dropped/duplicated frames, or audio being stretched/re-encoded incorrectly. Here’s a practical way to debug it.

1) Determine the type of sync problem

First figure out how it’s out of sync:

• Constant offset: audio is always early/late by the same amount
  • Often caused by edit points, trimming, encoder delay, or muxing offsets
• Drift over time: starts in sync but gradually gets worse
  • Often caused by frame-rate mismatch, variable frame rate, or resampling issues
• Only certain segments are wrong
  • Often caused by dropped frames, cuts, or bad timestamps in source footage

2) Inspect source and output metadata

Check the properties of both the input and output files:

• Frame rate: fps
• Variable vs constant frame rate
• Audio sample rate: 44100, 48000, etc.
• Container timestamps / timebase
• Duration reported by the container and streams

If you’re using ffmpeg, ffprobe is the best tool:

ffprobe -hide_banner -show_streams -show_format input.mp4
ffprobe -hide_banner -show_streams -show_format output.mp4

Look for:

• r_frame_rate
• avg_frame_rate
• time_base
• start_time
• duration
• audio sample_rate

3) Check for variable frame rate issues

A very common cause is taking VFR footage (often from phones, screen recordings, OBS, etc.) and processing it as if it were constant frame rate.

Symptoms:

• Sync is okay at first, then drifts
• Different players show slightly different behavior

Fix:

• Normalize to constant frame rate before editing/processing, or ensure the pipeline preserves timestamps properly.

Example:

ffmpeg -i input.mp4 -vf fps=30 -af aresample=async=1 -c:v libx264 -c:a aac output.mp4

4) Compare durations carefully

Check whether the audio and video durations match expected runtime:

• If video is 10 seconds shorter/longer than audio, there may be a timestamp or trimming issue
• If output duration is wrong, muxing or concat steps may have altered stream timing

Commands:

ffprobe -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 input.mp4
ffprobe -show_entries stream=index,codec_type,duration,start_time -of compact input.mp4

5) Watch for frame drops/duplicates

If frames are dropped or duplicated during processing, video can speed up/slow down relative to audio.

Check logs from your processing tool:

• dropped frames
• duplicated frames
• “non-monotonous DTS”
• timestamp warnings

With ffmpeg, enable verbose logging:

ffmpeg -loglevel verbose -i input.mp4 -c:v libx264 -c:a aac output.mp4

6) Check for audio resampling or re-encoding problems

Audio sync can break if:

• sample rate changes unexpectedly
• audio is resampled without proper timestamp handling
• audio is encoded with encoder delay not accounted for

For example, AAC encoding can introduce priming samples/delay. Many players handle this, but some pipelines don’t.

If possible, try:

• copying audio without re-encoding to see if the issue persists:

ffmpeg -i input.mp4 -c:v libx264 -c:a copy test.mp4

If sync is fine with copied audio but broken when re-encoding, the issue is likely in your audio encode/resample step.

7) Verify edits/cuts are aligned to keyframes or timestamps

If you trim or concat videos, inaccurate cut points can create A/V offsets.

For concatenation:

• Ensure all segments have matching codecs, sample rates, frame rates, and timebases
• Prefer a concat demuxer or re-encode consistently

Example concat demuxer:

# list.txt
file 'part1.mp4'
file 'part2.mp4'

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

If streams don’t match exactly, -c copy may produce sync problems. In that case, re-encode.

8) Use a reference point to measure offset

Pick a visible and audible event:

• a clap
• a flash
• a spoken word

Then compare:

• when the flash occurs in video
• when the sound occurs in audio

If you can quantify the offset at multiple timestamps, you can tell whether it’s:

• fixed offset
• linear drift

9) Try remuxing vs re-encoding

Sometimes the issue is only in the container metadata. Try remuxing:

ffmpeg -i input.mp4 -c copy remuxed.mp4

If remuxed output is still out of sync, the issue is likely in the source timestamps or processing chain.

10) Common fixes in ffmpeg workflows

Depending on the issue:

For timestamp/VFR handling

ffmpeg -fflags +genpts -i input.mp4 -c:v libx264 -c:a aac output.mp4

For audio drift correction

ffmpeg -i input.mp4 -af aresample=async=1:first_pts=0 -c:v copy -c:a aac output.mp4

For forcing constant frame rate

ffmpeg -i input.mp4 -vf fps=30 -vsync cfr -c:v libx264 -c:a aac output.mp4

11) Reproduce the problem with a minimal sample

If the issue is in a larger pipeline:

• take a short clip where sync breaks
• run each processing step separately
• compare metadata after each step

This isolates the exact stage introducing the error.

12) If you’re writing code, inspect timestamps

If you’re using libraries like:

• FFmpeg
• GStreamer
• OpenCV
• MoviePy
• PyAV

make sure you’re not:

• assuming each frame is evenly spaced when it isn’t
• discarding PTS/DTS
• writing frames with incorrect timestamps
• mixing audio from one timebase with video from another

---

Quick debugging checklist

1. Compare input/output metadata with ffprobe
2. Check whether source is VFR
3. Determine if sync issue is constant offset or drift
4. Look for dropped/duplicated frames
5. Test remuxing vs re-encoding
6. Verify audio sample rate and resampling
7. Test with a short clip and isolate each step

If you want, I can also give you:

• a step-by-step ffmpeg debugging workflow
• help analyzing specific ffprobe output
• or a checklist for Python/OpenCV/MoviePy pipelines.