Mismatch between mediaPresentationDuration and total duration of periods could be handled better

[alkerway]

Environment

• The MPD passes the DASH-IF Conformance Tool on https://conformance.dashif.org/
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream
• The issue observed is not mentioned on https://github.com/Dash-Industry-Forum/dash.js/wiki/FAQ
• The issue occurs in the latest reference client on http://reference.dashif.org/dash.js/ and not just on my page

• Link to playable MPD file: https://aow.im/dash/manifest.mpd
• Dash.js version: 4.4.1 or any other
• Browser name/version: Chrome 100 or any other browser
• OS name/version: MacOS or any other os

Steps to reproduce

1. Play the multi-period manifest
2. Seek to a point near the end
3. Let the video play until the end

Observed behavior

The video contents ends, all periods play fully and successfully, but the current time never reaches the duration, dash.js hangs and the video element never emits the ended event

Console output

[6290][MediaPlayer] Streaming Initialized 
Debug.js:169 [6293][MediaPlayer] Playback Initialized 
Debug.js:169 [7052][DashParser] Parsing complete: ( xml2json: 36.3ms, objectiron: 1.80ms, total: 0.0381s) 
Debug.js:169 [7053][StreamController] Manifest updated... updating data system wide. 
Debug.js:169 [7063][ManifestUpdater] Manifest has been refreshed at Wed Jun 22 2022 12:12:03 GMT+1200 (New Zealand Standard Time)[1655856723.484]  
Debug.js:169 [7073][StreamController] Switch to stream 0. Seektime is 0, current playback time is null. Seamless period switch is set to false 
Debug.js:169 [7075][MediaSourceController] Set MediaSource duration:5798.952 
Debug.js:169 [7079][Stream] No muxed data. 
Debug.js:169 [7079][Stream] No image data. 
Debug.js:169 [7080][AbrController] Stream ID: 0 [video] switch from 0 to 5/5 (buffer: 0) . 
Debug.js:169 [7083][AbrController] Stream ID: 0 [audio] switch from 0 to 3/3 (buffer: 0) . 
Debug.js:169 [7219][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/init0.m4f  
Debug.js:169 [7220][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 0 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/init0.m4f  
Debug.js:169 [7221][ScheduleController][video] [video] lastInitializedRepresentationInfo changed to 5 
Debug.js:169 [7222][ScheduleController][audio] [audio] lastInitializedRepresentationInfo changed to 3 
Debug.js:169 [7223][PlaybackController] Native video element event: play 
Debug.js:169 [7224][PlaybackController] Native video element event: waiting 
Debug.js:169 [7224][PlaybackController] Native video element event: loadedmetadata 
Debug.js:169 [7225][ScheduleController][video] Top quality video index has changed from NaN to 5 
Debug.js:169 [7226][ScheduleController][audio] Top quality audio index has changed from NaN to 3 
Debug.js:169 [7590][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 0 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment0.m4f  
Debug.js:169 [7720][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment0.m4f  
Debug.js:169 [7723][PlaybackController] Requesting seek to time: 0.08 (internal) 
Debug.js:169 [7770][PlaybackController] Native video element event: seeked 
Debug.js:169 [7771][PlaybackController] Native video element event: loadeddata 
Debug.js:169 [7771][PlaybackController] Native video element event: playing 
Debug.js:169 [8059][StreamProcessor][text] OnFragmentLoadingCompleted for stream id 0 and media type text - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/a292ecf6-8d16-42b7-8fbc-8d1df2deec85/08a8e4b1-f7c5-45b2-8ca2-6ad47d8151d9.vtt  
Debug.js:169 [8355][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 0 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment1.m4f  
Debug.js:169 [8577][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment1.m4f  
Debug.js:169 [9087][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 0 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment2.m4f  
Debug.js:169 [9355][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment2.m4f  
document.querySelector('video').currentTime = document.querySelector('video').duration - 10
5788.952
Debug.js:169 [9786][PlaybackController] Seeking to: 5788.952 
Debug.js:169 [9787][StreamProcessor][video] onFragmentLoadingAbandoned request: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment3.m4f has been aborted 
Debug.js:169 [9787][StreamProcessor][audio] onFragmentLoadingAbandoned request: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment3.m4f has been aborted 
Debug.js:169 [9788][PlaybackController] Native video element event: waiting 
Debug.js:169 [9791][StreamController] Switch to stream 35. Seektime is 5788.952, current playback time is 5788.952. Seamless period switch is set to true 
Debug.js:169 [9794][Stream] No muxed data. 
Debug.js:169 [9794][Stream] No image data. 
Debug.js:169 [9795][AbrController] Stream ID: 35 [video] switch from 0 to 5/5 (buffer: 0) . 
Debug.js:169 [9799][AbrController] Stream ID: 35 [audio] switch from 0 to 3/3 (buffer: 0) . 
Debug.js:169 [9970][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/init35.m4f  
Debug.js:169 [9970][ScheduleController][audio] [audio] lastInitializedRepresentationInfo changed to 3 
Debug.js:169 [9971][ScheduleController][audio] Top quality audio index has changed from NaN to 3 
Debug.js:169 [10204][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/init35.m4f  
Debug.js:169 [10204][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment2932.m4f  
Debug.js:169 [10210][ScheduleController][video] [video] lastInitializedRepresentationInfo changed to 5 
Debug.js:169 [10212][ScheduleController][video] Top quality video index has changed from NaN to 5 
Debug.js:169 [10983][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment2933.m4f  
Debug.js:169 [11042][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment2898.m4f  
Debug.js:169 [11232][StreamProcessor][text] OnFragmentLoadingCompleted for stream id 35 and media type text - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/a292ecf6-8d16-42b7-8fbc-8d1df2deec85/08a8e4b1-f7c5-45b2-8ca2-6ad47d8151d9.vtt  
Debug.js:169 [11238][PlaybackController] Native video element event: seeked 
Debug.js:169 [11238][PlaybackController] Native video element event: playing 
Debug.js:169 [11666][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment2899.m4f  
Debug.js:169 [11970][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment2900.m4f  
Debug.js:169 [12163][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment2934.m4f  
Debug.js:169 [12164][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment2901.m4f  
Debug.js:169 [12304][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/segment2902.m4f  
Debug.js:169 [12311][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment2935.m4f  
Debug.js:169 [12448][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/segment2936.m4f  
Debug.js:169 [18858][PlaybackController] Native video element event: waiting 
Debug.js:169 [20827][GapController] Jumping to end of stream because of gap from 5796.649057 to 5796.74125. Gap duration: 0.09219300000040676 
doLog @ Debug.js:169
warn @ Debug.js:132
_jumpGap @ GapController.js:354
_onWallclockTimeUpdated @ GapController.js:182
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
_onWallclockTime @ PlaybackController.js:635
(anonymous) @ PlaybackController.js:484
setInterval (async)
startUpdatingWallclockTime @ PlaybackController.js:483
_onPlaybackStart @ PlaybackController.js:528
play (async)
play @ VideoModel.js:270
play @ PlaybackController.js:180
_onBufferLevelUpdated @ StreamController.js:716
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
_triggerEvent @ BufferController.js:1011
_updateBufferLevel @ BufferController.js:661
_onAppended @ BufferController.js:290
(anonymous) @ BufferController.js:250
Promise.then (async)
_appendToBuffer @ BufferController.js:249
_onInitFragmentLoaded @ BufferController.js:210
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
onFragmentLoadingCompleted @ FragmentController.js:154
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
onLoadingCompleted @ FragmentModel.js:284
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
report @ FragmentLoader.js:89
success @ FragmentLoader.js:116
onload @ HTTPLoader.js:229
load (async)
load @ XHRLoader.js:82
internalLoad @ HTTPLoader.js:311
load @ HTTPLoader.js:357
load @ URLLoader.js:67
load @ FragmentLoader.js:98
loadCurrentFragment @ FragmentModel.js:219
executeRequest @ FragmentModel.js:207
_onInitFragmentNeeded @ StreamProcessor.js:380
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
_getNextFragment @ ScheduleController.js:168
schedule @ ScheduleController.js:144
setTimeout (async)
startScheduleTimer @ ScheduleController.js:102
startScheduleControllers @ Stream.js:835
(anonymous) @ StreamController.js:486
Promise.then (async)
_activateStream @ StreamController.js:472
_onMediaSourceOpen @ StreamController.js:442
Debug.js:169 [20827][PlaybackController] Requesting seek to time: 5796.74125 
Debug.js:169 [20828][PlaybackController] Seeking to: 5796.74125 
Debug.js:169 [20829][PlaybackController] Native video element event: waiting 
Debug.js:169 [20949][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/init35.m4f  
Debug.js:169 [20950][ScheduleController][audio] [audio] lastInitializedRepresentationInfo changed to 3 
Debug.js:169 [20952][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/init35.m4f  
Debug.js:169 [20952][ScheduleController][video] [video] lastInitializedRepresentationInfo changed to 5 
Debug.js:169 [22824][GapController] Jumping to end of stream because of gap from 5796.661249 to 5796.74125. Gap duration: 0.08000100000026578 
doLog @ Debug.js:169
warn @ Debug.js:132
_jumpGap @ GapController.js:354
_onWallclockTimeUpdated @ GapController.js:182
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
_onWallclockTime @ PlaybackController.js:635
(anonymous) @ PlaybackController.js:484
setInterval (async)
startUpdatingWallclockTime @ PlaybackController.js:483
_onPlaybackStart @ PlaybackController.js:528
play (async)
play @ VideoModel.js:270
play @ PlaybackController.js:180
_onBufferLevelUpdated @ StreamController.js:716
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
_triggerEvent @ BufferController.js:1011
_updateBufferLevel @ BufferController.js:661
_onAppended @ BufferController.js:290
(anonymous) @ BufferController.js:250
Promise.then (async)
_appendToBuffer @ BufferController.js:249
_onInitFragmentLoaded @ BufferController.js:210
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
onFragmentLoadingCompleted @ FragmentController.js:154
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
onLoadingCompleted @ FragmentModel.js:284
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
report @ FragmentLoader.js:89
success @ FragmentLoader.js:116
onload @ HTTPLoader.js:229
load (async)
load @ XHRLoader.js:82
internalLoad @ HTTPLoader.js:311
load @ HTTPLoader.js:357
load @ URLLoader.js:67
load @ FragmentLoader.js:98
loadCurrentFragment @ FragmentModel.js:219
executeRequest @ FragmentModel.js:207
_onInitFragmentNeeded @ StreamProcessor.js:380
(anonymous) @ EventBus.js:124
trigger @ EventBus.js:124
_getNextFragment @ ScheduleController.js:168
schedule @ ScheduleController.js:144
setTimeout (async)
startScheduleTimer @ ScheduleController.js:102
startScheduleControllers @ Stream.js:835
(anonymous) @ StreamController.js:486
Promise.then (async)
_activateStream @ StreamController.js:472
_onMediaSourceOpen @ StreamController.js:442
Debug.js:169 [22824][PlaybackController] Requesting seek to time: 5796.74125 
Debug.js:169 [22824][PlaybackController] Seeking to: 5796.74125 
Debug.js:169 [22957][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 35 and media type video - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/4460760524001_water2000_3_1280x720/init35.m4f  
Debug.js:169 [22958][ScheduleController][video] [video] lastInitializedRepresentationInfo changed to 5 
Debug.js:169 [22960][StreamProcessor][audio] OnFragmentLoadingCompleted for stream id 35 and media type audio - Url: https://redirector.playback.ap-southeast-2.prod.deploys.brightcove.com/playback/once/redirect/v1/4460760524001/08c131e6-4cb9-445e-8bff-bf7db11bf97d/xb9/default_audio192_2_en_main/init35.m4f  
Debug.js:169 [22961][ScheduleController][audio] [audio] lastInitializedRepresentationInfo changed to 3 

Expected behavior

When the last period ends dash.js should end the video and the video element should send out the ended event.

This is not an issue with dash.js because the manifest is generated incorrectly. However, the bitmovin player handles the incorrectly generated manifest and the incorrectly generated manifests are generated by Brightcove so the issue is not too obscure. It might make the dash.js library more robust to add a patch as long as the patch is reasonably small and clean.

What I have now for a patch is the following 7 lines in ManifestLoader.js, but maybe there is a better way.

I would like to download share the .mpd file but would need to get client permission and may only be able to share it privately. If it is worth making a change I will work on getting the manifest and submitting a pull request.

If it is not worth fixing feel free to close this issue. Thank you for any help!

[dsilhavy]

@alkerway Thanks for the detailed description and your solution. Your approach looks reasonable, still one question: In your application are you relying on the native video element to fire the ended event? We do have a manual check if the playback has ended in dash.js that uses the duration of the last period. However I dont see the corresponding log output in your description:

logger.info('onPlaybackEnded -- PLAYBACK_ENDED but native video element didn\'t fire ended');

Can you please check if we dispatch the PLAYBACK_ENDED event for your content.

[alkerway]

@dsilhavy Thanks, you pointed me in exactly the right direction.

The period correctly fires the STREAM_BUFFERING_COMPLETED event but the PLAYBACK_ENDED event is not dispatched because the last period isn't recognised as the last period - its isLast attribute is false when it should be true.

Tracing it back further, it seems the isLast attribute is set by checking if the period start + duration is within a second of the manifest mediaPresentationDuration. In the case of the streams I am testing with the difference is usually more than a second and can be up to 4-5.

My thought would be to change that check to determine whether a period is the last one by comparing its id attribute with the last period's id attribute. The below diff shows what I mean:

However, it looks like if a period does not explicitly define an id attribute (though dash.js still generates one), there will be no id attribute on the period accessed from within the period.mpd.manifest.Periods array, so this fix doesn't work with manifests that don't explicitly define period id attributes.

If there is a good way around that let me know, otherwise I have submitted a pull request with the original patch I proposed that overwrites the mediaPresentationDuration attribute of the manifest with the sum of all periods' durations. If mediaPresentationDuration is overwritten, the difference between the last period's start + duration and the manifest's duration will be zero, which is within the threshold to determine if the period is the last period.

Pull request here: #3971

I was able to download and share a stream from the client if you would like to test, it can be accessed at https://aow.im/dash/manifest.mpd