Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Write colr atom to muxed MP4 when present in input AV1 file #1251

Conversation

CaitlinOCallaghan
Copy link
Contributor

This PR is an extension of the full AV1 codec string feature: PR 1205 and relates to Issue 1007 and Issue 1202.

As per the AV1 spec, the codec string may contain optional color values. These color values are critical for detecting HDR video streams - see Issue 1007. Color information is extracted from the input mp4's colr atom and used to generate the full AV1 codec string. This PR preserves the color information by writing the colr atom to the muxed mp4.

References:

Testing

Manual: AV1 video WITH colr atom

Note: Presence of the colr atom in bbbhdr_av1.mp4 was verified with a hex editor.
image

  1. Generate Packager MPD with: ./out/Release/packager in=~/Downloads/bbbhdr_av1.mp4,stream=video,output=av1_with_colr.mp4 --mpd_output av1_with_colr.mpd
  2. Check manifest generated by Shaka Packager for Representation codecs string:
<?xml version="1.0" encoding="UTF-8"?>
<!--Generated with https://github.com/shaka-project/shaka-packager version 31129ee-release-->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 DASH-MPD.xsd" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" minBufferTime="PT2S" type="static" mediaPresentationDuration="PT1.100000023841858S">
  <Period id="0">
    <AdaptationSet id="0" contentType="video" width="640" height="360" frameRate="15360/512" subsegmentAlignment="true" par="16:9">
      <Representation id="0" bandwidth="778146" codecs="av01.1.01M.10.0.000.09.16.09.0" mimeType="video/mp4" sar="1:1">
        <BaseURL>av1_with_colr.mp4</BaseURL>
        <SegmentBase indexRange="808-851" timescale="15360">
          <Initialization range="0-807"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>
  1. Check av1_with_colr.mp4 bitstream for the colr atom and cross reference the values of the atom's properties with the codec representation string found in the manifest:
    image

Format

git clang-format --style Chromium --binary /usr/bin/clang-format 80e024013df87a4bfeb265c8ea83cfa2a0c5db0f

@CaitlinOCallaghan CaitlinOCallaghan changed the title Cocallaghan/writing colr atom feat: Write colr atom to muxed MP4 when present in input AV1 file Jul 27, 2023
uint16_t transfer_characteristics,
uint16_t matrix_coefficients,
uint8_t video_full_range_flag) const {
return base::StringPrintf(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A link to https://aomediacodec.github.io/av1-isobmff/#codecsparam here would be really helpful, IMO

Copy link
Contributor Author

@CaitlinOCallaghan CaitlinOCallaghan Aug 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link is included above on line 76 - I think the default PR review port cut it off:
image

@cosmin
Copy link
Contributor

cosmin commented Aug 4, 2023

Now that #1205 this needs to be rebased

@CaitlinOCallaghan
Copy link
Contributor Author

Rebased! :) @cosmin

@@ -429,6 +430,36 @@ bool MP4Muxer::GenerateVideoTrak(const VideoStreamInfo* video_info,
VideoSampleEntry video;
video.format =
CodecToFourCC(video_info->codec(), video_info->h26x_stream_format());
if (video.format == FOURCC_av01) {
auto av1_color_info =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

having to parse these back out of the codec string doesn't feel like the right solution

@CaitlinOCallaghan
Copy link
Contributor Author

Re-published as PR#1261, with an improved and more stable implementation.

@github-actions github-actions bot added the status: archived Archived and locked; will not be updated label Oct 28, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status: archived Archived and locked; will not be updated
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants