Support new Periods adding new encryption info

[TheJohnBowers]

Have you read the FAQ and checked for duplicate open issues?:
Yes, I didn't see anything that looked like a duplicate issue

What version of Shaka Player are you using?:
2.3 built from master

Can you reproduce the issue with our latest release version?:
It also occurs when building 2.3 from the v2.3.x branch

Can you reproduce the issue with the latest code from master?:
Yes

Are you using the demo app or your own custom app?:
Demo App

If custom app, can you reproduce the issue using our demo app?:
NA

What browser and OS are you using?:
Chrome 64 on Mac OS -- Also Firefox Quantum 58.02 on MacOS

What are the manifest and license server URIs?:

{
  name: 'Downlynk Dash Live with Ad Breaks - duseqa1',
  manifestUri: 'https://content-duseqa1.downlynk.com/channel/29e9c85c070f413fb71a3b2a39eed208.mpd?ad=dashad', // eslint-disable-line max-len
  encoder: shakaAssets.Encoder.UPLYNK,
  source: shakaAssets.Source.UPLYNK,
  drm: [
    shakaAssets.KeySystem.WIDEVINE,
    shakaAssets.KeySystem.PLAYREADY
  ],
  features: [
    shakaAssets.Feature.MP4,
    shakaAssets.Feature.PSSH,
    shakaAssets.Feature.MULTIKEY,
    shakaAssets.Feature.HIGH_DEFINITION
  ],
  licenseServers: {
    'com.microsoft.playready': 'https://content-duseqa1.downlynk.com/pr',
    'com.widevine.alpha': 'https://content-duseqa1.downlynk.com/wv'
  },
  requestFilter: shakaAssets.UplynkRequestFilter,
  responseFilter: shakaAssets.UplynkResponseFilter
},

What did you do? / What did you expect to happen? / What actually happened?
The above test stream is running a sample ad that is 116 seconds in length every 5 minutes. If the live stream is joined during the drm protected content portion of the stream, everything works fine. Playback transitions into and out of the unencrypted ad portion perfectly over and over again.

IF however I just happen to join the stream during the ad portion when the manifest contains no drm protected period, playback stalls when a period eventually pops into the live window with widevine content protection. Playback seems to proceed to the end of the unencrypted period, and then hang. A few segments of the new period seem to be downloaded an buffered, but playback doesn't proceed.

Firefox actually gives a nasty decoder error and stops, while chrome just kind of sits there. Here is the firefox error:

Shaka Error MEDIA.VIDEO_ERROR (3,,NS_ERROR_DOM_MEDIA_DECODE_ERR (0x806e0004) - mozilla::MediaResult mozilla::AppleATDecoder::DecodeSample(mozilla::MediaRawData *): Error decoding audio sample: 1852797029 @ 117269001)

The network pane doesn't show any attempt to request the widevine license when the protected content shows up in the live window, or at any other time during playback in this scenario. If I reload the stream once the live window again advertises protected content, playback works fine.

This is an important scenario to work if someone wants to use unencrypted/unprotected content for ads as you cannot be sure when viewers will join a live event.

I am not seeing this issue with ExoPlayer

[chrisfillmore]

This looks similar to #1094.

[TheJohnBowers]

It looks exactly like #1094 actually. I hadn't noticed that issue as it was closed. If I read that issue correctly, a fix was applied to setup the EME if license server config suggested that protected content would be provided. I do have license servers setup in the player config, so the provided fix should fix my issue? The EME log does show some activity when I start during an unencrypted ad period, but still no license is requested when the protected period shows up. I update master just this morning to ensure I had everything. So I have the code that is supposed to provide the fix, yet this issue persists.

eme_start_on_drm_protected_asset.log
eme_start_on_unencrypted_asset.log

[TheJohnBowers]

Just checked out the v2.3.x branch and verified the same issue occurs there also.

Perhaps any new license is a problem. I create a new test live stream that starts with encrypted content and then every 3 minutes a new period with new encryption keys starts. This would require a new license to be requested. I am watching the network pane, and I am sure that never happens. Because of this playback dies at the period boundary. It basically looks the same as when the live stream starts on a unencrypted period. So perhaps this has nothing specifically to do with starting on a unencrypted period, just a failure to request new licenses as they are required.

{
  name: 'Live New Asset / New Key Every 3 Min',
  manifestUri: 'https://content-duseqa1.downlynk.com/channel/40b7f8553f6a41a7a1c00b0147b5618a.mpd', // eslint-disable-line max-len
  encoder: shakaAssets.Encoder.UPLYNK,
  source: shakaAssets.Source.UPLYNK,
  drm: [
    shakaAssets.KeySystem.WIDEVINE,
    shakaAssets.KeySystem.PLAYREADY
  ],
  features: [
    shakaAssets.Feature.MP4,
    shakaAssets.Feature.PSSH,
    shakaAssets.Feature.MULTIKEY,
    shakaAssets.Feature.HIGH_DEFINITION
  ],
  licenseServers: {
    'com.microsoft.playready': 'https://content-duseqa1.downlynk.com/pr',
    'com.widevine.alpha': 'https://content-duseqa1.downlynk.com/wv'
  },
  requestFilter: shakaAssets.UplynkRequestFilter,
  responseFilter: shakaAssets.UplynkResponseFilter
},

[TheModMaker]

The problem is that we currently only support key rotation when there are no pssh data in the manifest. See #1107 for some more info. The way we support key rotation is by listening for encrypted events from the browser when it sees a pssh box in the media. We use this to send the license request. When the pssh data is in the manifest, we only use that and ignore encrypted events. But right now, we don't look for new pssh data in the new Periods. So when we get a new Period, we should look for new pssh data and handle them.

[chrisfillmore]

@TheModMaker is this ticket a duplicate of #1107 now?

[TheJohnBowers]

The PSSH data on these streams only exists in the manifest, and not in the init segments at all. So in that sense it is slightly different than what seems to be describe in #1107. Taking the PSSH data out of the manifest wouldn't solve anything as appears to be the case with 1107. Other than that, it does look like it would be a very similar issue.

[joeyparrish]

We will look into supporting new init data from manifest updates. Sorry for the trouble!

[TheJohnBowers]

Is this on the radar anywhere? Scheduled to be looked at for any particular targeted release, etc?

[TheModMaker]

@TheJohnBowers This is scheduled for v2.5, which will be our next feature release. We will be working on this soon, but no one is working on it right now.

[TheJohnBowers]

So -- I just pulled from master and built this morning, but It doesn't seem to be working quite right -- When the key change comes up I do see this in the console log:
VM1663:418 emeLogger.CreateSessionCall {timestamp: "11:0:49:804", title: "CreateSessionCall",
target: MediaKeys, returned: e…r.MediaKeySession, sessionType: "temporary"}
VM1663:418 emeLogger.GenerateRequestCall {timestamp: "11:0:49:804", title:
"GenerateRequestCall", target: e…r.MediaKeySession, returned: Promise, initDataType: "cenc", …}
VM1663:418 emeLogger.CreateSessionCall {timestamp: "11:0:49:805", title: "CreateSessionCall",
target: MediaKeys, returned: e…r.MediaKeySession, sessionType: "temporary"}
VM1663:418 emeLogger.GenerateRequestCall {timestamp: "11:0:49:807", title:
"GenerateRequestCall", target: e…r.MediaKeySession, returned: Promise, initDataType: "cenc", …}
VM1663:418 emeLogger.PromiseResult {timestamp: "11:0:49:810", title: "GenerateRequestCall
Promise Result", status: "resolved", args: Array(2)}
VM1663:418 emeLogger.PromiseResult {timestamp: "11:0:49:813", title: "GenerateRequestCall
Promise Result", status: "resolved", args: Array(2)}
VM1663:418 emeLogger.MessageEvent {title: "MessageEvent", event: MediaKeyMessageEvent,
timestamp: "17:20:7:398", target: e…r.MediaKeySession, message: Uint8Array(2), …}
VM1663:418 emeLogger.UpdateCall {timestamp: "11:0:50:38", title: "UpdateCall", target: e…
r.MediaKeySession, returned: Promise, response: Uint8Array(716), …}
VM1663:418 emeLogger.PromiseResult {timestamp: "11:0:50:151", title: "UpdateCall Promise
Result", status: "resolved", args: Array(1)}
VM1663:418 emeLogger.MessageEvent {title: "MessageEvent", event: MediaKeyMessageEvent,
timestamp: "17:20:7:738", target: e…r.MediaKeySession, message: Uint8Array(4036), …}
VM1663:418 emeLogger.UpdateCall {timestamp: "11:0:50:368", title: "UpdateCall", target: e…
r.MediaKeySession, returned: Promise, response: Uint8Array(569), …}
VM1663:418 emeLogger.PromiseResult {timestamp: "11:0:50:433", title: "UpdateCall Promise
Result", status: "resolved", args: Array(1)}
VM1663:418 emeLogger.KeyStatusesChangeEvent {title: "KeyStatusesChangeEvent", event:
Event, timestamp: "17:20:8:19", target: e…r.MediaKeySession}

But video hangs at the point the key change occurs. Interestingly if I wait for a bit the audio does start playing again (when the playhead is moved to the back of the TSB), but video never recovers. Is it possible that only the audio key was peroperly rotated?

This is the stream I tested it on this morning. Key rotates every 3 minutes:

https://content.uplynk.com/channel/3ff06a21737f40bd9f0e3bc342a4ca6b.mpd?prettydash=1
https://content.uplynk.com/wv

[joeyparrish]

Thanks, @TheJohnBowers. We'll take another look.

[joeyparrish]

The work we've done so far was cherry-picked to v2.4.5. I said v2.5.0 before, but I wasn't reading my own notes when I was cherry-picking things. Whoops. Enjoy!

[TheJohnBowers]

So, I was able to create a couple of test streams that demonstrate adding new encryption info successfully and unsuccessfully when moving from unencrypted to encrypted in Shaka player. I figured you guys would be interested in seeing it, as it has been hit and miss for me whether this transition works:

These streams are the same live stream, but I start with a different preroll sample ad (unencrypted) in each case. With one specific sample ad I see Shaka report Dropping Variant (not compatible with key system, platform, or active Variant) (for every single variant) in the console when the first encrypted period shows up from the test content. The demo app then reports "Unplayable Period" and playback soon fails.

manifest URL: https://content.uplynk.com/channel/3c367669a83b4cdab20cceefac253684.mpd?ad=koolaid&drm_optional=1&ad.preroll=1&prettydash=1

license URL: https://content.uplynk.com/wv

Now If I just start with a different ad:

manifest URL: https://content.uplynk.com/channel/3c367669a83b4cdab20cceefac253684.mpd?ad=onead&drm_optional=1&ad.preroll=1&prettydash=1

license URL: https://content.uplynk.com/wv

Starting with this ad does almost the same thing when the first encrypted period shows up:

Dropping Variant (not compatible with key system, platform, or active Variant) shows up, but not for every variant. It drops most of them, but leaves a couple so that the next period is not considered unplayable, and the playback successfully continues.

Why is Shaka dropping variants when moving from unencrypted to encrypted periods, and why does it sometimes leave some variants when it moves from unencrypted to encrypted?

This seems relevant to this particular bug, so I thought I would post this info here, but perhaps it deserves a different bug.

One note on the test streams -- These streams have random ad breaks in them, so if you happen to start with a preroll ad in an ad break you will see the ad twice in a row. If you see the sample ad twice in a row you need to restart playback to duplicate the issue. I have no way to ensure that you aren't in an ad break when you start, so occasionally you won't be able to replicate - Just be aware.

[theodab]

The first manifest you provided changes codecs between periods. Notably, the two periods have no video codecs in common. This causes problems with our drm engine, which at the moment cannot add new codecs to its list of supported codecs after initialization. We have an issue for this, #1567.

Also, switching between different codecs isn't possible on most platforms. There's a new changeType method for this, but it's not universally available yet, nor do we support it at the moment. See #1528.

[TheJohnBowers]

I feel like different avc1 codecs strings wouldn't count as codec switching for the MSE just as joey mentioned here that different mp4a codecs strings wouldn't count.

#1567 (comment)

It isn't like we are switching from avc1 to hevc or something. The issue does seem to be covered by #1567 though.

[TheJohnBowers]

So I am not seeing the problem I saw when I first tested your fix. The problem of video hanging where the key change occurred. I have tested this in 2.5 and 2.4, and I think it is fixed. The other issue I thought might have been related appears to have its own bug #1567. I think this one can be closed as fixed.