Add Dolby Vision Transcoding and Editing Support #1235

ybai001 · 2024-04-02T05:01:08Z

This code contribution adds Dolby Vision transcoding and editing support to transformer.

Merge the code from androidx/media main branch

SheenaChhabra · 2024-05-09T11:25:43Z

@ybai001 I just wanted to update that due to other high priority work, it might take a few weeks to review this change. I appreciate your patience.

ybai001 · 2024-05-09T23:53:53Z

@ybai001 I just wanted to update that due to other high priority work, it might take a few weeks to review this change. I appreciate your patience.

Sure, understood.

SheenaChhabra · 2024-06-17T12:16:38Z

libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java

@@ -135,6 +135,9 @@ public DefaultCodec createForVideoDecoding(
    if (SDK_INT >= 31 && requestSdrToneMapping) {
      mediaFormat.setInteger(
          MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_SDR_VIDEO);
+    } else if (SDK_INT >= 31 && MimeTypes.VIDEO_DOLBY_VISION.equals(format.sampleMimeType)) {


We generally write the condition in reversed way format.sampleMimeType.equals(MimeTypes.VIDEO_DOLBY_VISION)

There's lots of occurrences of the 'yoda condition' in our codebase, but I agree with @SheenaChhabra it is now discouraged in new code. In this case, because Format.sampleMimeType is @Nullable you will need to use Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION).

SheenaChhabra · 2024-06-17T12:19:13Z

libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java

@@ -135,6 +135,9 @@ public DefaultCodec createForVideoDecoding(
    if (SDK_INT >= 31 && requestSdrToneMapping) {
      mediaFormat.setInteger(
          MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_SDR_VIDEO);
+    } else if (SDK_INT >= 31 && MimeTypes.VIDEO_DOLBY_VISION.equals(format.sampleMimeType)) {


Can you share some documentation which explains this code?

Dolby Vision is not only a HDR type but also a media type. But it is based on H.264, H.265 or AV1. That is to say, the Dolby Vision stream is base encoded data + Dolby Vision metadata. Dolby Vision decoder will do two things in playback use case: (1) Call base codec, e.g. H.265 decoder, to decode the base encoded data; (2) Apply Dolby Vision metadata to decoded base data (output of step 1) to get Dolby Vision experience.

In editing use case, Dolby Vision decoder does step 1 only. We don't apply the Dolby Vision metadata because the result of this process is related to the screen characteristic and environment brightness. For editing use case, the Dolby Vision metadata will be generated by Dolby Vision encoder. For transcoding use case, this is a trade off between better quality and consistent result on different devices.

By default, Dolby Vision decoder will do both above step 1 & 2. The purpose of
mediaFormat.setInteger(MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_HLG);
is to notify Dolby Vision decoder to do step 1 only.

There is no open specification to explain above logic. But I can share some documentations about Dolby Vision.

Dolby Vision official site: https://www.dolby.com/technologies/dolby-vision/

Dolby Vision specification and white paper: https://professionalsupport.dolby.com/s/specifications-and-white-papers?language=en_US

For transcoding use case, this is a trade off between better quality and consistent result on different devices.

Which trade off you are talking about?

For transcoding use case, this is a trade off between better quality and consistent result on different devices.

Which trade off you are talking about?

For transcoding use case, ideally, we should do above step 1 & 2 for decoding then encoding to another video format. This is "better quality". In practice (in this contribution), we do step 1 only for decoding. The reason is the result of step 2 is related to the screen characteristic and environment brightness. That means user will get different results on different devices. The transcoding result is not "consistent". So "ignoring step 2" is a trade off.

I am still not sure of this change. if the decoder givens us dynamic metadata then I think the pipeline will simple ignore it.

Can you try your code after removing this code and see how it works? As per my knowledge Transformer should ignore the additional metadata.

Also the transformer already supports decoding of DV so I wonder why we need any changes in the decoder factory.

This comment seems unresolved.

This is important for DV editing/transcoding use case. As explained in my first comment,

Dolby Vision is not only a HDR type but also a media type. But it is based on H.264, H.265 or AV1. That is to say, the Dolby Vision stream is base encoded data + Dolby Vision metadata. Dolby Vision decoder will do two things in playback use case: (1) Call base codec, e.g. H.265 decoder, to decode the base encoded data; (2) Apply Dolby Vision metadata to decoded base data (output of step 1) to get Dolby Vision experience.

In editing use case, Dolby Vision decoder does step 1 only. We don't apply the Dolby Vision metadata because the result of this process is related to the screen characteristic and environment brightness. For editing use case, the Dolby Vision metadata will be generated by Dolby Vision encoder. For transcoding use case, this is a trade off between better quality and consistent result on different devices.

By default, Dolby Vision decoder will do both above step 1 & 2. This is correct for playback use case.
But for editing/transcoding use case, Dolby Vision decoder to do step 1 only. (ignoring Dolby Vision metadata.) That is the purpose of
mediaFormat.setInteger(MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_HLG);
which notifies Dolby Vision decoder to do step 1 only.

Does this answer your question?

SheenaChhabra · 2024-06-17T12:21:09Z

libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java

@@ -108,6 +111,11 @@ public int addTrack(Format format) throws MuxerException {
    if (isVideo) {
      mediaFormat = MediaFormat.createVideoFormat(sampleMimeType, format.width, format.height);
      MediaFormatUtil.maybeSetColorInfo(mediaFormat, format.colorInfo);
+      if (sampleMimeType.equals(MimeTypes.VIDEO_DOLBY_VISION) &&
+          android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.TIRAMISU) {
+        mediaFormat.setInteger(MediaFormat.KEY_PROFILE, getDvProfile(format));


Can you please share some corresponding spec which explains these changes?

As far as Dolby Vision and Profile, the spec is here: https://professionalsupport.dolby.com/s/article/What-is-Dolby-Vision-Profile?language=en_US

And I'd like to add some explanation why I did this code change:

In Media3 library code FrameworkerMuxer.java, addTrack(Format format) method, it calls system MediaMuxer.addTrack(mediaFormat) method without PROFILE/LEVEL settings.

This is OK for other video formats but results in issue for Dolby Vision. The key point is in Android AOSP utils.cpp file, convertMessageToMetadata() method, KEY_PROFILE is checked. If there is no this parameter, BAD_VALUE will be returned.

see: https://cs.android.com/android/platform/superproject/main/+/main:frameworks/av/media/libstagefright/Utils.cpp;drc=216a97bdad3c0f492094250b33d82abf27098932;l=2115

SheenaChhabra

We need to have some tests cases around these changes.

ybai001 · 2024-06-18T05:39:25Z

We need to have some tests cases around these changes.

What can I do for this request? At least, I can provide test stream. If you mean I need to implement some test cases, could you guide me what kinds of test cases I should add. Previously, I added some DD+JOC and AC-4 test cases for my media3 contributions so that I am familiar with that process. But for transformer, this is my first contribution.

…boratories/media into dlb/dovi-transformer/dev

SheenaChhabra · 2024-06-26T15:54:14Z

We need to have some tests cases around these changes.

What can I do for this request? At least, I can provide test stream. If you mean I need to implement some test cases, could you guide me what kinds of test cases I should add. Previously, I added some DD+JOC and AC-4 test cases for my media3 contributions so that I am familiar with that process. But for transformer, this is my first contribution.

We could include following tests:

Transmuxing a dolby vision input file should preserve the dolby vision data. This could be a robolectric test. For this, you can refer tests in MediaItemExportTest.java
Transcoding a normal H.264/H.265 to dolby vision. For this your can refer tests in TransformerEndToEndTest.java
Editing a dolby vision input. For this your can refer tests in TransformerEndToEndTest.java

I would also suggest to run these scenario using Transformer demo app. This is to ensure that it works as expected.

ybai001 · 2024-07-02T02:45:26Z

We need to have some tests cases around these changes.

What can I do for this request? At least, I can provide test stream. If you mean I need to implement some test cases, could you guide me what kinds of test cases I should add. Previously, I added some DD+JOC and AC-4 test cases for my media3 contributions so that I am familiar with that process. But for transformer, this is my first contribution.

We could include following tests:

Transmuxing a dolby vision input file should preserve the dolby vision data. This could be a robolectric test. For this, you can refer tests in MediaItemExportTest.java

Transcoding a normal H.264/H.265 to dolby vision. For this your can refer tests in TransformerEndToEndTest.java

Editing a dolby vision input. For this your can refer tests in TransformerEndToEndTest.java

I would also suggest to run these scenario using Transformer demo app. This is to ensure that it works as expected.

Hi, @SheenaChhabra,
Thanks for your suggestion. As far as the three tests you mentioned,

I checked the file MediaItemExportTest.java and there are several test cases for transmux. But all of them are general test cases. None of them is for specific video format, e.g. AV1. Could you help to clarify why we need to add test case for Dolby Vision but not for other video format? And which test case I should refer to? Instead, I find there is a case in https://github.com/androidx/media/blob/main/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java#L79. Should I create a case for Dolby Vision here?
I have added the case as you mentioned. I created it based on this one (for AV1) https://github.com/androidx/media/blob/main/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java#L1517
I checked the file TransformerEndToEndTest.java. But none of them is related to HDR editing. Instead, I find there is a Dolby Vision editing case in https://github.com/androidx/media/blob/main/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java#L191. Is it enough for your requirement?

Finally,
(1) I added export_transmuxDolbyVisionFile() in HdrEditingTest.java to match your requirement 1;
(2) I added transcode_withOutputVideoMimeTypeDolbyVision_completesSuccessfully() in TransformerEndToEndTest.java to match your requirement 2;
(3) I applied exist exportAndTranscode_dolbyVisionFile_whenHdrEditingIsSupported() and created exportAndTranscode_dolbyVisionFile_whenHdrEditingUnsupported_toneMapsOrThrows() in HdrEditingTest.java to match your requirement 3

If this solution is OK, I'll upload my code.

ybai001 · 2024-07-03T09:02:29Z

I have uploaded the test cases code.

SheenaChhabra · 2024-07-25T14:32:09Z

...es/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java

@@ -1280,6 +1280,39 @@ public void transcode_withOutputVideoMimeTypeAv1_completesSuccessfully() throws
    assertThat(exportResult.videoMimeType).isEqualTo(MimeTypes.VIDEO_AV1);
  }

+  @Test
+  public void transcode_withOutputVideoMimeTypeDolbyVision_completesSuccessfully() throws Exception {


On which device you have tested this which can produce dolby vision metadata?

OPPO flagship series (e.g OPPO Find X6/X7 pro/ultra), ViVo flagship series (e.g. ViVo X90/pro/ultra, X100/pro/ultra) and XiaoMi flagship series.
I tested it on my Pixel phone with Dolby Vision codec integration.

SheenaChhabra · 2024-07-25T14:34:26Z

...es/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java

+            .run(testId, editedMediaItem);
+    ExportResult exportResult = exportTestResult.exportResult;
+
+    assert exportTestResult.filePath != null;


I think its not needed.

I added this assert sentence because Android Studio had a warning message. I can remove it if you are fine.

I'll remove it based on your latest comment.

SheenaChhabra · 2024-07-25T14:40:04Z

libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java

+      level = MediaCodecInfo.CodecProfileLevel.DolbyVisionLevelFhd24;  // Level 03
+    } else if (maxWidthHeight <= 2560 && pps <= 62208000) {
+      level = MediaCodecInfo.CodecProfileLevel.DolbyVisionLevelFhd30;  // Level 04
+    } else if (maxWidthHeight <= 3840) {


Can we use this existing method?

https://source.corp.google.com/piper///depot/google3/third_party/java_src/android_libs/media/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtil.java;l=307?q=MediaCodecUti#:~:text=306-,307,-308

It's for different purpose. The method in your link is to PARSE the Dolby Vision profile and level from codec string. The use case is to play a streaming content. You can get codec string from a MPEG-DASH manifest file. Then paring this codec string, you will get profile/level info. This info can be used to select which Dolby Vision decoder to handle this streaming content (H.264 based Dolby Vision decoder or H.265 based Dolby Vision decoder). In this case, Format.codecs is a valid codec string.

As far as this code contribution case, it is used to decide which Dolby Vision profile/level should be used based on incoming content width/height/framerate. In this case. Format.codecs is NULL.

SheenaChhabra · 2024-07-25T14:42:44Z

libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java

@@ -108,6 +111,11 @@ public int addTrack(Format format) throws MuxerException {
    if (isVideo) {
      mediaFormat = MediaFormat.createVideoFormat(sampleMimeType, format.width, format.height);
      MediaFormatUtil.maybeSetColorInfo(mediaFormat, format.colorInfo);
+      if (sampleMimeType.equals(MimeTypes.VIDEO_DOLBY_VISION) &&
+          android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.TIRAMISU) {


Can we change it to pattern similar to SDK_INT >= 33 to make it consistent with other code in this class.

SheenaChhabra · 2024-07-25T14:47:45Z

libraries/transformer/src/main/java/androidx/media3/transformer/EncoderUtil.java

+      case MimeTypes.VIDEO_DOLBY_VISION:
+        if (colorTransfer == C.COLOR_TRANSFER_HLG) {
+          return ImmutableList.of(
+              MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvheSt);


Out of all profile, why have we chosen only this one?

Good question. All exist devices which support Dolby Vision encoding on market will encode the input stream to this profile only. In future, maybe we will extend this list. But Dolby has not finalized which profiles we should support for encoding.

SheenaChhabra · 2024-07-25T14:55:56Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

@@ -139,6 +139,41 @@ public void export_transmuxHlg10File() throws Exception {
    assertThat(actualColorTransfer).isEqualTo(C.COLOR_TRANSFER_HLG);
  }

+  @Test
+  public void export_transmuxDolbyVisionFile() throws Exception {


I think transmuxing test case can also go into MediaItemExportTest as it does not need to run on an actual device.

Could you give me more hint about it?

I added the Dolby Vision test case here because there are export_transmuxHdr10File() and export_transmuxHlg10File() test cases in this file. I did the similar thing in my test case implementation.

In MediaItemExportTest, I found some transmux test cases but I didn't find corresponding test cases for HLG and HDR10. Do I need to add test case for Dolby Vision in this file?

I think these tests needs to run on real device because in Robolectric tests, FrameworkMuxer does not write to the actual output file and we are doing validations on the actual output file. So we should keep the test case here only. Thanks!

SheenaChhabra · 2024-07-25T15:03:46Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+
+    if (Util.SDK_INT < 24) {
+      // TODO: b/285543404 - Remove suppression once we can transmux H.265/HEVC before API 24.
+      recordTestSkipped(context, testId, /* reason= */ "Can't transmux H.265/HEVC before API 24");


The below check for format support should check whether muxer supports this format or not so this is not needed.

I just did the same thing as export_transmuxHdr10File() and export_transmuxHlg10File(). Shall we remove above code in these three methods together?

I am not sure why existing tests are written in that way but as per my understanding for transmuxing decoder/encoder should not get involved. So the below check for decoder should not be there. We should definitely check for muxer, which is done separately here and should remain.

You can leave the existing test as it is.

I think this is not fully resolved as the decoder check (AndroidTestUtil.skipAndLogIfFormatsUnsupported) is still present which should not be required for transmuxing.

OK, I'll remove this part of checking.

if (AndroidTestUtil.skipAndLogIfFormatsUnsupported( context, testId, /* inputFormat= */ MP4_ASSET_DOLBY_VISION_HDR_FORMAT, /* outputFormat= */ null)) { return; }

SheenaChhabra · 2024-07-25T15:06:14Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+        new TransformerAndroidTestRunner.Builder(context, transformer)
+            .build()
+            .run(testId, mediaItem);
+    assert exportTestResult.filePath != null;


We generally use assertThat(xyz).isNotNull();

OK, I'll change it.

Based on your latest comment, I'll remove this sentence.

SheenaChhabra · 2024-07-25T15:06:58Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+    assert exportTestResult.filePath != null;
+    @C.ColorTransfer
+    int actualColorTransfer =
+        Objects.requireNonNull(


I think we can remove it.

SheenaChhabra · 2024-07-25T15:07:14Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+            retrieveTrackFormat(context, exportTestResult.filePath, C.TRACK_TYPE_VIDEO)
+                .colorInfo)
+            .colorTransfer;
+    assertThat(actualColorTransfer).isEqualTo(C.COLOR_TRANSFER_HLG);


Can we check the mime type as well?

SheenaChhabra · 2024-07-25T15:07:54Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

    @C.ColorTransfer
    int actualColorTransfer =
-        retrieveTrackFormat(context, exportTestResult.filePath, C.TRACK_TYPE_VIDEO)
-            .colorInfo
+        Objects.requireNonNull(


Why do we need these changes?

Just because there is a warning message from latest Android Studio. I can revert this change if you are fine.

TLDR; you can remove these and other null checks from the test code.

Just had a discussion with my team about these warnings and we have decided to ignore these. This is because we have other internal tools to check these nullness issue (although we have disabled them for test to avoid null check overhead).

I'll remove this checking as you latest comment.

SheenaChhabra · 2024-07-26T10:48:54Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+    assert format.colorInfo != null;
+    if (deviceSupportsHdrEditing(VIDEO_H265, format.colorInfo)) {
+      recordTestSkipped(context, testId, /* reason= */ "Device supports Dolby Vision editing.");
+      return;


We can possibly use assumeDeviceDoesNotSupportHdrEditing() ?

Sorry, I'm not familiar with resolving confliction on GitHub. Let's complete other issues then handle the confliction issue. (I fork the main branch at the beginning of this code contribution. At that time, there is no this method.)

SheenaChhabra · 2024-07-26T10:53:30Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+    }
+
+    if (AndroidTestUtil.skipAndLogIfFormatsUnsupported(
+        context, testId, /* inputFormat= */ format, /* outputFormat= */ null)) {


This method has been renamed to assumeFormatsSupported(). Sorry for the conflicts.

SheenaChhabra · 2024-07-26T10:53:55Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+            .addListener(
+                new Transformer.Listener() {
+                  @Override
+                  public void onFallbackApplied(


This method is deprecated, use the alternative.

This is still not resolved.

OK, I'll change the code to use

default void onFallbackApplied( Composition composition, TransformationRequest originalTransformationRequest, TransformationRequest fallbackTransformationRequest) { MediaItem mediaItem = composition.sequences.get(0).editedMediaItems.get(0).mediaItem; onFallbackApplied(mediaItem, originalTransformationRequest, fallbackTransformationRequest); }

SheenaChhabra · 2024-08-02T17:23:57Z

...es/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java

+    String actualMimeType =
+        retrieveTrackFormat(context, exportTestResult.filePath, C.TRACK_TYPE_VIDEO).sampleMimeType;
+    assertThat(actualMimeType).isEqualTo(MimeTypes.VIDEO_DOLBY_VISION);
+    assertThat(exportResult.exportException).isNull();


If there is any exception then test would actually throw so this check is not required.

I just follow Line 1278 and 1331 in transcode_withOutputVideoMimeTypeAv1_completesSuccessfully() and transcode_withOutputAudioMimeTypeAac_completesSuccessfully()

I am not sure why existing tests has this assertion but we can remove it for the newly added tests.

OK, I'll remove it.

SheenaChhabra · 2024-08-02T17:24:49Z

...es/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java

+    assertThat(actualMimeType).isEqualTo(MimeTypes.VIDEO_DOLBY_VISION);
+    assertThat(exportResult.exportException).isNull();
+    assertThat(exportResult.durationMs).isGreaterThan(0);
+    assertThat(exportResult.videoMimeType).isEqualTo(MimeTypes.VIDEO_DOLBY_VISION);


Since we are checking the actual mime type, this check can be removed.

I just followed Line 1280 and 1333 in transcode_withOutputVideoMimeTypeAv1_completesSuccessfully() and transcode_withOutputAudioMimeTypeAac_completesSuccessfully() respectively.

SheenaChhabra · 2024-08-02T17:24:54Z

...es/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java

+    String actualMimeType =
+        retrieveTrackFormat(context, exportTestResult.filePath, C.TRACK_TYPE_VIDEO).sampleMimeType;
+    assertThat(actualMimeType).isEqualTo(MimeTypes.VIDEO_DOLBY_VISION);
+    assertThat(exportResult.exportException).isNull();


If there is any exception then test would actually throw so this check is not required.

I just followed Line 1278 and 1331 in transcode_withOutputVideoMimeTypeAv1_completesSuccessfully() and transcode_withOutputAudioMimeTypeAac_completesSuccessfully() respectively.

SheenaChhabra · 2024-08-02T17:30:20Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+
+    if (Util.SDK_INT < 24) {
+      // TODO: b/285543404 - Remove suppression once we can transmux H.265/HEVC before API 24.
+      recordTestSkipped(context, testId, /* reason= */ "Can't transmux H.265/HEVC before API 24");


I am not sure why existing tests are written in that way but as per my understanding for transmuxing decoder/encoder should not get involved. So the below check for decoder should not be there. We should definitely check for muxer, which is done separately here and should remain.

You can leave the existing test as it is.

SheenaChhabra · 2024-08-02T17:31:47Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+  public void export_transmuxDolbyVisionFile() throws Exception {
+    Context context = ApplicationProvider.getApplicationContext();
+
+    if (Util.SDK_INT < 24) {


We have now started using assumeTrue(Util.SDK_INT >= 24). By using this the test is marked as skipped instead of marked passed.
So the whole if block check can be replaced with assumeTrue(Util.SDK_INT >= 24)

OK, I'll change the code.

You can also use @SdkSuppress(minSdkVersion = 24) if your only condition is SDK level.

https://developer.android.com/reference/androidx/test/filters/SdkSuppress

This is preferable to assumeTrue imo because it skips the test "earlier", and just doesn't run it at all, instead of needing to mark it as 'skipped' or 'passed' (which has mixed support in different test running environments).

OK, I think this method is better. I'll change the code again.

SheenaChhabra · 2024-08-02T17:35:38Z

libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java

@@ -135,6 +135,9 @@ public DefaultCodec createForVideoDecoding(
    if (SDK_INT >= 31 && requestSdrToneMapping) {
      mediaFormat.setInteger(
          MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_SDR_VIDEO);
+    } else if (SDK_INT >= 31 && MimeTypes.VIDEO_DOLBY_VISION.equals(format.sampleMimeType)) {


Can you try your code after removing this code and see how it works? As per my knowledge Transformer should ignore the additional metadata.

SheenaChhabra · 2024-08-02T17:38:48Z

libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java

+  @RequiresApi(33)
+  private static int getDvProfile(Format format) {
+    // Currently, only profile 8 is supported for encoding
+    // TODO: set profile ID based on format.


What is the work involved for this TODO?

Currently, only profile 8 (H.265 based) is supported for encoding. In future, maybe Dolby will support other Dolby Vision profile (e.g. profile 9/10). It's not finalized. If that is true, we need to add logic here to decide which profile should be used based on the parameter "format".

SheenaChhabra · 2024-08-02T17:48:51Z

libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java

@@ -135,6 +135,9 @@ public DefaultCodec createForVideoDecoding(
    if (SDK_INT >= 31 && requestSdrToneMapping) {
      mediaFormat.setInteger(
          MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_SDR_VIDEO);
+    } else if (SDK_INT >= 31 && MimeTypes.VIDEO_DOLBY_VISION.equals(format.sampleMimeType)) {


Also the transformer already supports decoding of DV so I wonder why we need any changes in the decoder factory.

SheenaChhabra · 2024-08-02T18:10:34Z

libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java

+    } else if (maxWidthHeight <= 1920 && pps <= 49766400) {
+      level = MediaCodecInfo.CodecProfileLevel.DolbyVisionLevelFhd24;  // Level 03
+    } else if (maxWidthHeight <= 2560 && pps <= 62208000) {
+      level = MediaCodecInfo.CodecProfileLevel.DolbyVisionLevelFhd30;  // Level 04


Do you know why the value of these constants is not same as mentioned level? For example DolbyVisionLevelFhd30 = 8

Because the codec profile/level constants should be a bitmask, which is defined by Android AOSP. Dolby defined our profile/level to follow this rule. See https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/media/java/android/media/MediaCodecInfo.java;drc=6a4bef0f90e822e19866e53a98b85029bff04ea0;l=4301

I think Google defined it in this way due to efficiency. For example, to judge whether one profile is supported by current device, you just need to do one time comparison:
if (appointedLevel & (DolbyVisionLevelFhd24 | DolbyVisionLevelFhd24) {
// Current device support this level
} else {
// Current device doesn't support this level
}

SheenaChhabra · 2024-08-02T18:12:51Z

libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java

+  }
+
+  // Get Dolby Vision level
+  // Refer to https://professionalsupport.dolby.com/s/article/What-is-Dolby-Vision-Profile


Please write it as a java doc comment. Same comment for other method.

SheenaChhabra · 2024-08-02T18:16:27Z

@ychaparov Can you please help with another review of this PR?

ychaparov · 2024-08-05T11:28:37Z

Looks good to me. @SheenaChhabra please approve when you think your comments have been resolved

ybai001 · 2024-08-06T03:23:24Z

@SheenaChhabra, I have updated code as your comments except resolving conflicts. Let me know whether there is any other comment from your side.

SheenaChhabra

Overall changes are good but some of the comments are still not resolved.
Once the comments are resolved/answered and the conflicts are resolved, we can take this forward.

SheenaChhabra · 2024-08-07T15:36:29Z

...es/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java

+    String actualMimeType =
+        retrieveTrackFormat(context, exportTestResult.filePath, C.TRACK_TYPE_VIDEO).sampleMimeType;
+    assertThat(actualMimeType).isEqualTo(MimeTypes.VIDEO_DOLBY_VISION);
+    assertThat(exportResult.exportException).isNull();


I am not sure why existing tests has this assertion but we can remove it for the newly added tests.

SheenaChhabra · 2024-08-07T15:40:48Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+
+    if (Util.SDK_INT < 24) {
+      // TODO: b/285543404 - Remove suppression once we can transmux H.265/HEVC before API 24.
+      recordTestSkipped(context, testId, /* reason= */ "Can't transmux H.265/HEVC before API 24");


I think this is not fully resolved as the decoder check (AndroidTestUtil.skipAndLogIfFormatsUnsupported) is still present which should not be required for transmuxing.

SheenaChhabra · 2024-08-07T15:42:24Z

libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/HdrEditingTest.java

+            .addListener(
+                new Transformer.Listener() {
+                  @Override
+                  public void onFallbackApplied(


This is still not resolved.

SheenaChhabra · 2024-08-07T15:43:26Z

libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java

@@ -135,6 +135,9 @@ public DefaultCodec createForVideoDecoding(
    if (SDK_INT >= 31 && requestSdrToneMapping) {
      mediaFormat.setInteger(
          MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_SDR_VIDEO);
+    } else if (SDK_INT >= 31 && MimeTypes.VIDEO_DOLBY_VISION.equals(format.sampleMimeType)) {


This comment seems unresolved.

ybai001 · 2024-08-08T06:16:19Z

Overall changes are good but some of the comments are still not resolved. Once the comments are resolved/answered and the conflicts are resolved, we can take this forward.

I have updated the code as you requested and resolved the conflicts. Let's me know whether there is any other issue.

ybai001 · 2024-08-08T06:59:53Z

After resolving the merge conflict, I find the code base is updated to the latest one and I have to update my test code to match the latest one. (Some definitions are changed totally, e.g. MP4_ASSET.
I'll refine my code and upload it.

ybai001 · 2024-08-08T09:28:59Z

@SheenaChhabra, I updated code based on latest code on main branch. I think I resolved all the conflicts. Let me know whether you have more questions. Sorry for tons of commits and comments today.

SheenaChhabra

Thanks for making this change.
I'm going to send this for internal review now. You may see some more commits being added as I make changes in response to review feedback. Please refrain from pushing any more substantive changes as it will complicate the internal review - thanks!

ybai001 · 2024-08-12T05:58:09Z

Thanks for making this change. I'm going to send this for internal review now. You may see some more commits being added as I make changes in response to review feedback. Please refrain from pushing any more substantive changes as it will complicate the internal review - thanks!

Thanks. BTW, just want to let you know:
Google can't push changes to this organization-owned forks because Dolby didn't grant you collaborator access. I have applied for it to our Dolby admin but she didn't agree with it. As result, you can merge this PR but it will result in an 'evil' merge. See details here: https://github.com/androidx/media/blob/release/CONTRIBUTING.md#push-access-to-pr-branches

ybai001 added 2 commits March 18, 2024 11:11

Merge pull request #4 from androidx/main

563f4d8

Merge the code from androidx/media main branch

Add Dolby Vision Transcoding and Editing Support

a5524c0

rohitjoins requested review from droid-girl and removed request for droid-girl April 3, 2024 12:17

rohitjoins assigned droid-girl Apr 3, 2024

droid-girl assigned SheenaChhabra and unassigned droid-girl Apr 8, 2024

droid-girl requested a review from SheenaChhabra April 16, 2024 12:50

adarsh-dhakad approved these changes May 4, 2024

View reviewed changes

SheenaChhabra reviewed Jun 17, 2024

View reviewed changes

ybai001 added 2 commits June 18, 2024 13:56

Add Dolby Vision Transcoding and Editing Support

6acfa0c

Merge branch 'dlb/dovi-transformer/dev' of https://github.com/DolbyLa…

7b09e6c

…boratories/media into dlb/dovi-transformer/dev

Add test cases for Dolby Vision transformer support

5eb064d

ybai001 mentioned this pull request Jul 4, 2024

Should Media3 use media duration to calculate frame rate rather than track duration? #1531

Closed

satmandu mentioned this pull request Jul 17, 2024

[Issue]: 4k DV HDR10 HEVC black screen during playback pixel 6a jellyfin/jellyfin-android#1371

Open

3 tasks

SheenaChhabra reviewed Jul 25, 2024

View reviewed changes

SheenaChhabra reviewed Jul 26, 2024

View reviewed changes

Update code based on Google comments

6b6d66c

SheenaChhabra reviewed Aug 2, 2024

View reviewed changes

Update test code based on Google comments

2822919

ybai001 added 2 commits August 6, 2024 11:09

Replace assumeTrue() with @SdkSupress()

d8321cd

Remove unused imported class

fd13345

SheenaChhabra requested changes Aug 7, 2024

View reviewed changes

ybai001 added 3 commits August 8, 2024 11:13

Update code again based on latest comments

b8d02d9

Fix a bug in HdrEditingTest.java

ffecefb

Merge branch 'main' into dlb/dovi-transformer/dev

0f43554

Refine code to match the latest branch

62e1383

SheenaChhabra approved these changes Aug 9, 2024

View reviewed changes

SheenaChhabra added the should merge label Aug 15, 2024

Add Dolby Vision Transcoding and Editing Support #1235

Are you sure you want to change the base?

Add Dolby Vision Transcoding and Editing Support #1235

Conversation

ybai001 commented Apr 2, 2024

SheenaChhabra commented May 9, 2024

ybai001 commented May 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SheenaChhabra left a comment

Choose a reason for hiding this comment

ybai001 commented Jun 18, 2024

SheenaChhabra commented Jun 26, 2024

ybai001 commented Jul 2, 2024 • edited Loading

ybai001 commented Jul 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SheenaChhabra Jul 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SheenaChhabra Jul 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ybai001 commented Jul 2, 2024 •

edited

Loading

SheenaChhabra Jul 25, 2024 •

edited

Loading

SheenaChhabra Jul 26, 2024 •

edited

Loading