Support CEA-608/708 on DASH

[avelad]

Have you read the FAQ and checked for duplicate open issues?: Yes

What version of Shaka Player are you using?: 2.3.5

Can you reproduce the issue with our latest release version?: Yes

Can you reproduce the issue with the latest code from master?: Yes

Are you using the demo app or your own custom app?: Demo app

What browser and OS are you using?: Amy

What are the manifest and license server URIs?:
VOD CEA-608:
https://vm2.dashif.org/dash/vod/testpic_2s/cea608.mpd
Live CEA-608:
https://vm2.dashif.org/livesim/testpic_2s/cea608.mpd
VOD CEA-608 + TTML
https://vm2.dashif.org/dash/vod/testpic_2s/cea608_and_segs.mpd
Live CEA-608 + TTML
https://vm2.dashif.org/livesim/testpic_2s/cea608_and_segs.mpd

According to https://dashif.org/docs/DASH-IF-IOP-v4.2-clean.pdf

6.4.3.3. Signaling CEA-608 caption service metadata

The Accessibility descriptor shall be provided with @schemeIdUri set to urn:scte:dash:cc:cea-608:2015, and an optional @value attribute to describe the captions. If the @value attribute is not present, the Representation contains a CEA-608 based closed captioning service.
DASH-IF Interoperability Points v4.1 141
If present, the @value attribute shall contain a description of caption service(s) provided in the stream as a list of channel-language pairs. Alternatively, a simple list of language codes may be provided, but this is strongly discouraged as it will not provide sufficient information to map the language with the appropriate caption channel.
The @value syntax shall be as described in the ABNF below.
@value = (channel *3 [";" channel]) / (language *3[";" language])
channel = channel-number "=" language
channel-number = CC1 | CC2 | CC3 | CC4
language = 3ALPHA ; language code per ISO 639.2/B

Examples:
Simple signaling of presence of CEA-608 based closed caption service (Note: Not signaling languages is a discouraged practice)
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-608:2015"/>
Signaling of presence of CEA-608 closed caption service languages in English and German
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-608:2015" value="eng;deu"/>
Signaling of presence of CEA-608 closed caption service in English and German, with channel assignments
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-608:2015" value="CC1=eng;CC3=deu"/>
Signaling of presence of CEA-708 closed caption service in English and German
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-708:2015" value="1=lang:eng;2=lang:deu"/>
Signaling of presence of CEA-708 closed caption service in English and easy reader English
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-708:2015" value="1=lang:eng;2=lang:eng,war:1,er:1"/>

Note: In DASH-IF player, it works (http://reference.dashif.org/dash.js/v2.6.7/samples/dash-if-reference-player/index.html)

[avelad]

Sample of cea608 parser: https://github.com/Dash-Industry-Forum/dash.js/blob/development/externals/cea608-parser.js

[joeyparrish]

We currently only support CEA608/708 in TS segments, so we would need to investigate how to extract them from MP4 segments. Also, our current CEA608/708 support is based on the parser provided by mux.js. We would need to investigate whether or not it makes sense for mux.js to do this for MP4, as well.

[forbesjo]

We're actually planning this quarter on parsing fmp4 608/708 in mux.js in order to handle fmp4 HLS and DASH. Stay tuned!

[avelad]

@forbesjo, Do you have in mind to support multiple CC in the same MP4?

[forbesjo]

@avelad yes that should be supported

[joeyparrish]

Blocked on videojs/mux.js#190

[chrisfillmore]

@avelad @joeyparrish Are either of you aware of whether DASH specifies how subtitles should be prioritized, if DASH content has both CEA-608/708 in the video stream, but also side-loaded text tracks?

I would assume that the presence of side-loaded text tracks means that they should be prioritized over those embedded in the streams. But section 6.4.5 of IOP (Guidelines for side-loaded TTML and WebVTT files) does not specify.

[joeyparrish]

@chrisfillmore, I'm not sure of the answer, but in Shaka Player, we give applications control over track selection.

[dsparacio]

@joeyparrish Hi Joey been a while hope all is well? Wondering if you guys are looking at https://github.com/Dash-Industry-Forum/cea608.js for implementation. HLS.js used this and of course we used it for Dash.js. Feel free to use this if it suites your needs!

Update
I guess I overlooked the mention on that url so you are aware. just wanted to make sure!

[joeyparrish]

@dsparacio, thanks for the link! We'll keep it mind if we decide to explore alternatives to mux.js.

[forbesjo]

This is the mux.js PR for reference videojs/mux.js#197

[avelad]

mux.js has released version 4.5.0 with support for CEA608 in mp4.

[ronenneumark]

@joeyparrish - on which shake player version supporting CEA608 in mp4 will be available?

[rbond]

Second that on 608 on mp4. We have many use cases for pulling captions in band.

[ronenneumark]

@rbond - can you share the supported in band cases?

[joeyparrish]

Hi all,

When we update mux.js, there will probably still be integration work to complete on our side. As a new feature, this would not appear until v2.5 at the earliest.

[TheJohnBowers]

Just want to leave a +1 on this one. We have two features we really hope make it into Shaka 2.5 and this is one of them. -- The other is the support for key rotation in the manifest.

[joeyparrish]

Hi everyone,

We are hard at work on this, and it is a priority for us for v2.5. As soon as we have something to show for it, we will release another v2.5 beta. The final v2.5.0 will wait on both this and src= support (which enables both progressive mp4 playback and FairPlay support on macOS and iOS).

[avelad]

I see that the issue is on backlog. @joeyparrish , can you change it to 2.5?

[joeyparrish]

Which issue?

[ismena]

@joeyparrish This one! :)
@avelad Let me do it :)

[TheJohnBowers]

I have another live test stream that you can validate against here if it is useful:
manifest url: https://content.downlynk.com/channel/34b1b0717d9d44f6809c89ab130a16bf.mpd
license server: https://content.downlynk.com/wv

[joeyparrish]

Thanks, @ismena! I miss the obvious sometimes. And thanks, @TheJohnBowers, for the content! Very helpful.

[joeyparrish]

The work we've done so far has been released in v2.5.0-beta2. There are still integration tests to write, plus we need to align our HLS CEA implementation with the DASH CEA implementation, and we need to deprecate one-off APIs for HLS CEA in favor of the tracks API we are using for DASH CEA.

[avelad]

@joeyparrish the configuration forceTransmuxTS (https://shaka-player-demo.appspot.com/docs/api/shakaExtern.html#.StreamingConfiguration) will be necessary after changes in your HLS parser?

[jakubvojacek]

Just tried to compile shaka player from master and failing to use generated shaka-player.compiled.js due to ReferenceError: muxjs is not defined. I know I could link mux.js library but shouldn't it work without the library? Just trying to play regular dash without closed captions.

I think the issue was introduced here bcd10c2#diff-7fe7cffd0a8a1684650ad9d4ef4afdbaR62

[vaage]

@jakubvojacek, you are right, mux.js should not be required. I just spoke with @michellezhuogg and we have a plan to address the issue and she is working on implementing it right now. Hopefully this won't not be an issue for much longer.

[jakubvojacek]

Thank you for the confirmation, was unsure whether I did not mess up the compilation somehow.

[joeyparrish]

@jakubvojacek, the muxjs reference appears to be fixed now.

[jakubvojacek]

Yes, it is fixed, thank you!