Muted segments getting completely skipped

[fisewugp]

Checklist

• I have checked the issue page for duplicates
• I am running the latest version (download here)

Edition

Windows GUI Application

So I downloaded a vod and when watching on my computer the muted segments get completely skipped and the video bugs a little bit to catch up with the skipped audio. Uploading the vod directly to youtube seems to make it work fine but when I edit in the chat and render the whole thing, the audio for the muted segments is skipped which makes it desynced to the video. Also, if that could help, I tried watching the vod in VLC and it worked perfectly but when visualising the audio it seems that the muted segments only pause the audio instead of it being silent.
I'm sorry if this is unclear and if it is feel free to ask me more info.

[ScrubN]

I have replicated the "audio pausing" by watching a muted stream with the windows photo app, when exporting the audio stream via FFmpeg ffmpeg -i ....mp4 ....mp3 -c copy, and when adding the video to the Blender VSE timeline (also uses FFmpeg I think).

This issue is kind of in the same boat as #1123; I'm not really sure there's anything I can do without reverting #1103, which fixed a few separate issues itself.

[Zibbp]

I haven't taken a look at the code but I believe the issue is the audio stream is in different position in muted versus non-muted segments.

Muted segments have the audio stream in position 2.

Input #0, mpegts, from '1-muted.ts':
  Duration: 00:00:10.00, start: 70.872000, bitrate: 725 kb/s
  Program 1
  Stream #0:0[0x102]: Data: timed_id3 (ID3  / 0x20334449)
  Stream #0:1[0x101]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 640x360, 30 fps, 30 tbr, 90k tbn, 2k tbc
  Stream #0:2[0x100]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 130 kb/s

Normal segments have the audio stream in position 0.

Input #0, mpegts, from '1680.ts':
  Duration: 00:00:10.01, start: 16860.872000, bitrate: 733 kb/s
  Program 1
  Stream #0:0[0x100]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 66 kb/s
  Stream #0:1[0x101]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 640x360, 30 fps, 30 tbr, 90k tbn, 2k tbc
  Stream #0:2[0x102]: Data: timed_id3 (ID3  / 0x20334449)

I switched the audio track positions for all muted segments.

ffmpeg -i 1-muted.ts -map 0:2 -map 0:1 -map 0:0 -c copy output.ts
...
...
$ ffprobe -hide_banner output.ts
Input #0, mpegts, from 'output.ts':
  Duration: 00:00:10.00, start: 1.467000, bitrate: 778 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 130 kb/s
  Stream #0:1[0x101]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 640x360, 30 fps, 30 tbr, 90k tbn, 2k tbc
  Stream #0:2[0x102]: Data: timed_id3 (ID3  / 0x20334449)

#!/bin/bash

for file in *-muted.ts; do
    if [ -f "$file" ]; then
        echo "Processing $file..."

        temp_file="temp_$file"

        ffmpeg -i "$file" -map 0:2 -map 0:1 -map 0:0 -c copy "$temp_file"

        if [ $? -eq 0 ]; then
            mv "$temp_file" "$file"
            echo "successfully processed and replaced $file"
        else
            echo "error processing $file"
            rm -f "$temp_file"
        fi
    fi
done

Concatenating the segments into a video results in a playable mp4 that has audio in non-muted segments, as expected. It also works in the Windows photo/video app.

ffmpeg -f concat -safe 0 -i concat.txt -c copy output.mp4

I've had nothing but issues with the audio from muted segments, especially from live streams with the 'ad is playing' screen.

[ScrubN]

I haven't taken a look at the code but I believe the issue is the audio stream is in different position in muted versus non-muted segments.

Nice catch! I saw that the muted segments still contained the audio streams, but completely missed that the order was different.

[ScrubN]

I just tested it locally (in the ffconcat list so preprocessing is not needed) and can confirm that audio works correctly in the windows photos app and blender!!

ffconcat solution

Also as a bonus, #1123 is also fixed by this solution exactly as I suspected!

I owe you a huge thanks @Zibbp for noticing that!

[ScrubN]

I will be releasing the fix for this soon, but I wanted to see about improving clip quality handling first (properly support best/source and worst keywords, etc.)

[fisewugp]

omg thank you so much!