Stuck buffering while tunneling. #6407

stevemayhew · 2019-09-06T19:22:07Z

This pull fixes the issue where the video render does not correctly report ready in tunneled mode. The specific bug is #6366

I can remove the manifest and network security changes (these allow local http testing on my workstation). I've provided a stream that reproduces this on streaming devices with Broadcom SOC chips.

The issue is largely timing related, streams that have very low video frame rate compared to the audio will cause the issue to manifest strait away.

stevemayhew

This is clearly a bug (Bug #6366 ) in the tunneling implementation. Can Andrew or someone else review this Thanks.

demos/main/src/main/AndroidManifest.xml

stevemayhew · 2019-09-06T19:25:18Z

demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java

@@ -365,6 +368,17 @@ private void initializePlayer() {
      RenderersFactory renderersFactory =
          ((DemoApplication) getApplication()).buildRenderersFactory(preferExtensionDecoders);

+      boolean enableTunneling = intent.getBooleanExtra(ENABLE_TUNNELED_PLAYBACK, false);
+      // Get a builder with current parameters then set/clear tunnling based on the intent


Add a simple intent extra (-ez enable_tunneled_playback true) that allows you view a URL with tunneling on for platforms that support it.

A similar option was implemented in 9af8511

Please rebase.

stevemayhew · 2019-09-06T19:25:56Z

library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecRenderer.java

+  protected boolean hasOutputReady() {
+    return hasOutputBuffer();
+  }
+


Here's the crux of the fix, the comments should spell out the intentions.

We can review and debate a good name...

stevemayhew · 2019-09-06T19:26:34Z

library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecRenderer.java

@@ -1362,7 +1376,7 @@ public boolean isReady() {
    return inputFormat != null
        && !waitingForKeys
        && (isSourceReady()
-            || hasOutputBuffer()
+            || hasOutputReady()


More descriptive of what the check really is.

stevemayhew · 2019-09-06T19:27:56Z

library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java

+  protected boolean hasOutputReady() {
+    return tunneling && buffersInCodecCount > 0 || super.hasOutputReady();
+  }
+


As the comment says, in tunneling mode once we have queued buffers to the video codec to decode it will render it when the matching audio syncs up.

The bug occurs because the audio track is stopped before the audio matching the video PTS is queued thus stalling the video codec forever.

andrewlewis · 2019-09-22T21:03:57Z

Had a quick look. This looks good for API 23 onwards where we can update buffersInCodecCount correctly. I'm not certain it works for API 21 though because we don't know when a buffer is rendered so we decrement the counter right after queueing (please let me know if I'm missing something).

stevemayhew · 2019-09-23T02:37:45Z

🙁it’s a damm shame Lollipop still even matters... Is tunneling even supported that old? But, yes, the fix won’t work correctly if the codec doesn’t report rendered events when in tunneled mode

…

On Sun, Sep 22, 2019 at 2:04 PM Andrew Lewis ***@***.***> wrote: Had a quick look. This looks good for API 23 onwards where we can update buffersInCodecCount correctly. I'm not certain it works for API 21 though because we don't know when a buffer is rendered so we decrement the counter right after queueing (please let me know if I'm missing something). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#6407?email_source=notifications&email_token=AADBF6GDP7B3CFP3CPKGXDLQK7MW3A5CNFSM4IUMI2I2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7JPFMQ#issuecomment-533918386>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADBF6HENP5HOPKW7J2HXX3QK7MW3ANCNFSM4IUMI2IQ> .

stevemayhew · 2019-10-25T18:20:42Z

I pulled out the changes not required for the actual bug fix, sorry for polluting the pull request with these.

andrewlewis · 2019-11-08T10:32:01Z

Thinking about this more, I'm not sure it's safe to assume we get one output buffer per input buffer when using tunneling mode, due to the possibility of the OEM doing frame rate conversion (this wouldn't be valid for non-tunneled playbacks and I think is covered by CTS, but as I understand it one of the reasons that tunneling mode exists is to give flexibility in postprocessing). I will investigate further whether we can avoid using buffersInCodecCount.

stevemayhew · 2020-03-13T18:42:29Z

@andrewlewis We are seeing this issue on boxes when there is low bandwidth that delays the audio (so similar to the issue the Music Choice reproduces).

One of my team members, @dpolyakova is our internal expert on all things Broadcom media stack and has some ideas on this.

googlebot · 2020-03-30T23:41:49Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

In tunneled playback mode the video renderer is potentially 'ready' with output at anytime there are buffers queued to the codec. This fix add the more specific boolean `hasOuputReady()` and uses this in the `isReady()` check. This changes fixes issue google#6366 The call to dequeueOutputBuffer() simply returns TRY_AGAIN always in tunneled mode (this comment is correct: google#1688 (comment) so the super.hasOutputReady() is always false in tunneling mode, even though the codec may have output ready.

…or. This allows player to go into buffering state before video playback starts freezing.

googlebot · 2020-04-01T21:11:39Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

dpolyakova · 2020-04-01T21:44:31Z

library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java

@@ -409,7 +415,15 @@ public boolean isReady() {
   */
  @Override
  protected boolean hasOutputReady() {
-    return tunneling && buffersInCodecCount > 0 || super.hasOutputReady();
+    boolean fifoReady = true;


Here we are trying to use FIFO length as renderer readiness criteria. If the frame rate is low such as 1fps on music channels the length would still be satisfactory and player wouldn't go into buffering state as long as there is a frame queued in the FIFO. If network bandwidth is low the FIFO length will get close to zero before decoder starts stalling. It's important to note that Broadcom's decoder will stall before rendering last few frames so zero FIFO length cannot be used as a criteria. The 333ms value is somewhat experimental. It was chosen to accommodate at least 10 frames of 30fps stream.

@andrewlewis @ojw28 Where are you guys at with addressing this issue. I'm preparing to update our code base to what will likely by your 2.11.5 release. We would very much like to not keep merging conflicted changes to the MediaCodecVideoRenderer, for obvious reasons.

Pretty sure we can send you an STB (Broadcom based) that reproduces the issue #6366 I think I included the URL for a section of music choice content that reproduces the issue 100% of the time

We are also see this, as Dasha explained, in other streams where audio is buffered sufficiently ahead of video. As I'm sure you guys are aware, Broadcom codecs often require some special spoon feeding.

Thanks again for looking at this with us, please tell me what I need to do to help the process along.

dpolyakova · 2020-04-01T21:47:33Z

@googlebot I consent

googlebot · 2020-04-01T21:47:36Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

googlebot added the cla: yes label Sep 6, 2019

stevemayhew mentioned this pull request Sep 6, 2019

Playback stalls forever in buffering with "Music Choice" content tunneling #6366

Open

stevemayhew commented Sep 17, 2019

View reviewed changes

andrewlewis self-assigned this Sep 20, 2019

andrewlewis assigned krocard and unassigned andrewlewis Nov 7, 2019

stevemayhew force-pushed the stuck-buffering-tunnel-issue branch from 502310d to d313601 Compare November 7, 2019 19:07

andrewlewis assigned andrewlewis and unassigned krocard Nov 8, 2019

andrewlewis added should merge and removed should merge labels Mar 5, 2020

stevemayhew force-pushed the stuck-buffering-tunnel-issue branch from d313601 to 01bf37e Compare March 14, 2020 01:38

googlebot added cla: no and removed cla: yes labels Mar 30, 2020

stevemayhew force-pushed the stuck-buffering-tunnel-issue branch from e82ce66 to 4096721 Compare March 30, 2020 23:46

stevemayhew force-pushed the stuck-buffering-tunnel-issue branch from 4096721 to a21e834 Compare March 30, 2020 23:53

Improved video stall detection by using queued FIFO length as indicat…

59fd033

…or. This allows player to go into buffering state before video playback starts freezing.

dpolyakova force-pushed the stuck-buffering-tunnel-issue branch from 818e5af to 59fd033 Compare April 1, 2020 21:19

dpolyakova reviewed Apr 1, 2020

View reviewed changes

googlebot added cla: yes and removed cla: no labels Apr 1, 2020

stevemayhew marked this pull request as draft February 25, 2022 17:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stuck buffering while tunneling. #6407

Stuck buffering while tunneling. #6407

stevemayhew commented Sep 6, 2019

stevemayhew left a comment

stevemayhew Sep 6, 2019

krocard Oct 25, 2019

stevemayhew Sep 6, 2019

stevemayhew Sep 6, 2019

stevemayhew Sep 6, 2019

stevemayhew Sep 6, 2019

andrewlewis commented Sep 22, 2019

stevemayhew commented Sep 23, 2019 via email

stevemayhew commented Oct 25, 2019

andrewlewis commented Nov 8, 2019

stevemayhew commented Mar 13, 2020

googlebot commented Mar 30, 2020

googlebot commented Apr 1, 2020

dpolyakova Apr 1, 2020

stevemayhew May 1, 2020

dpolyakova commented Apr 1, 2020

googlebot commented Apr 1, 2020

Stuck buffering while tunneling. #6407

Are you sure you want to change the base?

Stuck buffering while tunneling. #6407

Conversation

stevemayhew commented Sep 6, 2019

stevemayhew left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewlewis commented Sep 22, 2019

stevemayhew commented Sep 23, 2019 via email

stevemayhew commented Oct 25, 2019

andrewlewis commented Nov 8, 2019

stevemayhew commented Mar 13, 2020

googlebot commented Mar 30, 2020

googlebot commented Apr 1, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dpolyakova commented Apr 1, 2020

googlebot commented Apr 1, 2020