Improved generated test stream investigation

[defagos]

As a developer I want to have reliable local test streams with minimal footprint (i.e. files stored in the repo), covering future implementation needs:

• Alternative audio tracks.
• Alternative subtitles.
• Forced subtitles and AD tracks.
• Bonus: Sports streams (DVR turning into on-demand).

Acceptance criteria

• The described use cases have matching local test streams.
• Test stream generation without local files is investigated and implemented if possible.

Hints

From @amtins: See https://www.bogotobogo.com/FFMpeg/ffmpeg_video_test_patterns_src.php

Proposal for a time-displaying video:

ffplay -f lavfi -i testsrc=duration=10.1:size=1280x720:rate=15 -vf drawtext="fontfile=monofonto.ttf: fontsize=96: box=1: [email protected]: boxborderw=5: fontcolor=white: x=(w-text_w)/2: y=((h-text_h)/2)+((h-text_h)/4): text='timestamp: %{pts \: hms}'"

Tasks

• Investigate test stream generation without local files (see hints above), implement if appropriate and cleanup old files.
• Generate local test streams with alternative audio tracks.
• Generate local test streams with alternative subtitles.
• Bonus: If possible add example of a DVR stream which ends and turns into an on-demand stream.

[defagos]

We could also use a Docker image based on https://github.com/SRGSSR/docker-nginx-vod-module, as discussed with @bgaudaen.

One limitation in comparison to what we have is that the chunk size would be globally defined, whereas we are currently able to tweak it per sample stream.

[defagos]

Tools we could likely use as well:

• gpac
• Bento4
• Shaka packager
• Apple tools mediastreamsegmenter and variantplaylistcreator. This book can probably be helpful. The modern tool documentation, quite shorter, can be found here.

Below is a quick feedback. I also opened a (currently private) repository for investigation. This repository provides generation scripts. Generated content can then be served locally with

python3 -m http.server 8765 <directory>

Overall recommendation: Shaka Packager, as it supports most features we need to test.

gpac

Most definitely too obscure to use IMHO. Probably too powerful for our needs.

Bento4

Quite easy to use but lacks some features like support for FORCED subtitles or characteristics for AD. Could probably be added through a PR on mp4_dash.py, though, but requires to do the same work for DASH packaging as well.

Shaka packager

Quite easy to use but lacks support for FORCED subtitles. Supports characteristics with hls_characteristics, though, thus marking audio tracks for AD is possible. We could probably make a PR to add support for an hls_forced flag.

The language can be set with the lang descriptor. For a complete list of available descriptors refer to the source code.

[waliid]

How to add subtitles to a stream

Here is an example of how to add English and French subtitles to a stream.

Create subtitles files

SRT - SubRip subtitle with embedded timing

English

1
00:00:0,000 --> 00:00:1,000
Subtitle 1

2
00:00:1,000 --> 00:00:2,000 
Subtitle 2

3
00:00:2,000 --> 00:00:3,000
Subtitle 3

3
00:00:3,000 --> 00:00:4,000
Subtitle 4

French

1
00:00:0,000 --> 00:00:1,000
Sous-titre 1

2
00:00:1,000 --> 00:00:2,000 
Sous-titre 2

3
00:00:2,000 --> 00:00:3,000
Sous-titre 3

3
00:00:3,000 --> 00:00:4,000
Sous-titre 4

To convert to another format (vtt for example) we can use ffmpeg.
ffmpeg -i subtitles_en.srt Medias/subtitles_en.vtt

Merge subtitles to the stream

Generic context

ffmpeg -i <#video_path.mov#> -i <#path/subtitles_en.srt#> -i <#path/subtitles_fr.srt#> \
-map 0 -map 1 -map 2 \
-c:v copy -c:a copy -c:s mov_text \
-metadata:s:s:0 language=eng -metadata:s:s:1 language=fre \
<#output_path#>.mp4

Pillarbox context

ffmpeg -i Medias/nyan_cat.mov -i Medias/subtitles_en.srt -i Medias/subtitles_fr.srt \
-map 0:v:0 -map 0:a:0 -map 0:a:0 -map 1:s:0 -map 2:s:0 \
-c:v copy -c:a copy -c:s webvtt \
-metadata:s:s:0 language=eng -metadata:s:s:1 language=fre \
-metadata:s:a:0 language=eng -metadata:s:a:1 language=fre \
Medias/nyan_cat_subtitled.mkv

Deliver the stream via HLS

Pillarbox context

ffmpeg -stream_loop -1 \
-i "$MEDIAS_DIR/nyan_cat_subtitled.mkv" \
-c:s webvtt \
-map 0:v -map 0:a:0 -map 0:s:0 \
-f hls \
-var_stream_map "v:0,a:0,s:0,sgroup:subtitle,language:eng,name:english" \
-master_pl_name master.m3u8 \
-t 120 -hls_time 10 -hls_init_time 4 -hls_list_size 10 \
-master_pl_publish_rate 10 \
-hls_flags delete_segments+discont_start+split_by_time \
-hls_segment_filename "$ON_DEMAND_SUBTITLED_DIR/%v/seg%03d.ts" \
"$ON_DEMAND_SUBTITLED_DIR/%v/video.m3u8"

This approach works only for one subtitle track.

[defagos]

DVR to on-demand transition is probably not so useful for a unit testing setup since such a scenario requires the stream to be started, played for a while as a DVR stream before the stream changes to an on-demand one.

[defagos]

Work is being done to modernize how Shaka Packager is built using CMake. Currently migration has not reached a sufficient state so that we can efficiently work on adding forced subtitles support, though.