[MPEG-TS] Processing one frame videos fail

[zdanek]

System info

Operating System: Ubuntu focal 20.04
Shaka Packager Version: 2.6.1

Issue and steps to reproduce the problem

Packager does not handle one-frame mpeg-ts videos correctly. Frame duration is not calculated and any processing fails. Parser does not emit MediaSample with the video frame. Stream appears to be empty.

One-frame videos are valid. I'm working at live TV streaming provider. We're using AWS Elemental to generate mpeg-ts video stream that is then feed to our system. One-frame videos can occur when there's an ad break. Elemental closes current file and creates a new one with ad break video. Due to a coincidence sometimes it generates only one frame of TV program and then rest of the GOP goes to another file (with an ad video).

Shaka fails to process such one-video mpeg-ts frames.

Duration of a video frame is calculated by a subtraction of two consecutive DTS. If we have one frame we cannot calculate it.
If we have a Release version this is silenlty discarded but in Debug there's a DCHECK
https://github.com/google/shaka-packager/blob/master/packager/media/formats/mp2t/es_parser_h26x.cc#L102
that fails. This only the only symptom from command line. (We're using Shaka as a library to parse streams)

In Release version DCHECK does not block processing but as a deep consequence of pending_sample_duration_ == 0 the single video frame (MediaSample with video frame) is not emitted and a Video stream appears to be empty.

Packager Command:
We're using Shaka Packager as a library through an API but a small demo can be done even from cli.
I've prepared steps to reproduce the problem. We need ffmpeg, ffprobe and the Packager.
I assume that master is build with Release and Debug.
Examples should be run from source root.

Steps to reproduce the problem:

curl "https://upload.wikimedia.org/wikipedia/en/7/7d/Lenna_%28test_image%29.png" -o lena.png

ffmpeg -i lena.png -c:v libx264 -vf fps=25 -pix_fmt yuv420p out.ts

ffprobe -show_frames out.ts

notice that ffmpeg handles duration by heuristics

out/Release/packager input=out.ts --dump_stream_info

see that Shaka cannot get duration from mpeg-ts as it lacks of metadata such MP4 boxes have. Shaka does not sum up samples duration so it never show correct stream duration of a mpeg-ts file.

File "out.ts":
Found 1 stream(s).
Stream [0] type: Video
 codec_string: avc1.64001e
 time_scale: 90000
 duration: Infinite
 is_encrypted: false
 codec: H264
 width: 512
 height: 512
 pixel_aspect_ratio: 1:1
 trick_play_factor: 0
 nalu_length_size: 4

out/Debug/packager input=out.ts --dump_stream_info

it blows up due to mentioned DCHECK

File "out.ts":
Found 1 stream(s).
Stream [0] type: Video
 codec_string: avc1.64001e
 time_scale: 90000
 duration: Infinite
 is_encrypted: false
 codec: H264
 width: 512
 height: 512
 pixel_aspect_ratio: 1:1
 trick_play_factor: 0
 nalu_length_size: 4

[0104/185128:FATAL:es_parser_h26x.cc(102)] Check failed: pending_sample_duration_. 
#0 0x5644f56c2de5 base::debug::StackTrace::StackTrace()
#1 0x5644f5672153 logging::LogMessage::~LogMessage()
#2 0x5644f5ec1671 shaka::media::mp2t::EsParserH26x::Flush()
#3 0x5644f5ea1f36 shaka::media::mp2t::TsSectionPes::Flush()
#4 0x5644f5e8c29c shaka::media::mp2t::PidState::Flush()
#5 0x5644f5e8cb64 shaka::media::mp2t::Mp2tMediaParser::Flush()
#6 0x5644f5e7ff1c shaka::media::Demuxer::Parse()
#7 0x5644f5e7d478 shaka::media::Demuxer::Run()
#8 0x5644f5870895 shaka::media::Job::Run()
#9 0x5644f57b0cd6 base::SimpleThread::ThreadMain()
#10 0x5644f56a98c5 base::(anonymous namespace)::ThreadFunc()
#11 0x7f3badfdf609 start_thread
#12 0x7f3badf06293 clone

Aborted (core dumped)

What is the expected result?'
Emited MediaSample with video frame.
Packager does not blow up from cli.

What happens instead?

[0104/163319:FATAL:es_parser_h26x.cc(102)] Check failed: pending_sample_duration_. 
#0 0x555a6528fde5 base::debug::StackTrace::StackTrace()
#1 0x555a6523f153 logging::LogMessage::~LogMessage()
#2 0x555a65a8e671 shaka::media::mp2t::EsParserH26x::Flush()
#3 0x555a65a6ef36 shaka::media::mp2t::TsSectionPes::Flush()
#4 0x555a65a5929c shaka::media::mp2t::PidState::Flush()
#5 0x555a65a59b64 shaka::media::mp2t::Mp2tMediaParser::Flush()
#6 0x555a65a4cf1c shaka::media::Demuxer::Parse()
#7 0x555a65a4a478 shaka::media::Demuxer::Run()
#8 0x555a6543d895 shaka::media::Job::Run()
#9 0x555a6537dcd6 base::SimpleThread::ThreadMain()
#10 0x555a652768c5 base::(anonymous namespace)::ThreadFunc()
#11 0x7f02b7fba609 start_thread
#12 0x7f02b7ee1293 clone

Aborted (core dumped)

<Please attach the input files or email to [email protected].>

I can send you original one-frame video emited by AWS Elemental.
I have a PR to solve this problem.

[zdanek]

I've created a PR #1015

[zdanek]

Dear @kqyang or @joeyparrish can you look at this? This is my first PR so I'm thrilled if it will go through :)