Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add point and box geometry primivites and use them to create MediaReference bounding boxes #773

Conversation

rogernelson
Copy link
Collaborator

@rogernelson rogernelson commented Aug 18, 2020

Fixes #771
MediaReference: spatial bounding box

Summary

This change addresses the need to have spatial bounds expressed in the MediaReference. These are used to define a measurement-agnostic coordinate system representing the viewing area of the clip(s), which can be used by media players to determine how to display or scale/resize/center (if necessary) the media referenced by the Clips.

For a more in-depth explanation of the idea and motivations please see the issue linked here:
#771

I have flagged as many potential discussion points that I think might arise with Note below.

Note

  • I have my editor set to remove all trailing whitespace on save (since this is required by a lot of the Autodesk github linters which will block the PR if its not done). This resulted in some whitespace changes in the c++ code, Python was unaffected. If you'd like me to undo those, let me know. As an aside, a clang-format file in the repo root could be very useful so that coding style is enforced in c++ as well as Python.

Files changed

  • *.md files
    Lists the additions to the schemas (automatically generated by Makefiles).

  • src/opentime/CMakeLists.txt
    Adds new Point and Box files to the compilation

  • src/opentime/box.h
    Adds a new Box primitive with members width, height and center, including accessor and mutators for each. The default constructor, operator== and operator!= required by the serialization code is also added. Lastly, some common convenience methods are added: get_aspect_ratio, contains and get_union.
    The get_aspect_ratio method returns simply the ratio of width over height, commonly used by media applications. The contains method returns true if Point argument is contained within the Box. Lastly, the get_union method take a Box and returns the minimum sized box required to contain both Boxes. This is used by Composition/Timeline to calculate the union of all the Clips contained within them.
    Note: The contains and get_aspect_ratio methods are really only used by the unit tests, but I thought they were useful. I can remove it if you prefer not to have it.
    Note: To prevent a divide by 0, get_aspect_ratio returns 1 if the width is 0. If you'd prefer another default value, let me know.
    Python Bindings: Also added for all public functions.

  • src/opentime/point.h
    Adds a new Point primitive with members x and y, including accessor and mutators for each. The default constructor, operator== and operator!= required by the serialization code is also added.
    The class is mainly used by Box to define it's center.
    Note: I added a convenience templated is_equal method for floating point values here, that is also used by Box. If you'd prefer it to go somewhere else (like a utility file), let me know.
    Python Bindings: Also added for all public functions.

  • src/opentimelineio/clip.cpp
    Implement the bounds method. Simply returns the bounding box if it has been set on the clip or an error otherwise (conforming to the standard of the available_range method).

  • src/opentimelineio/clip.h
    Adds the bounds method to the public interface.
    Python Bindings: Also added

  • src/opentimelineio/composable.cpp
    Implements the bounds method to return an error so that any dervied classes that do not explicitly override the virtual method return the error (it is overriden by Clip, Stack and Track).

  • src/opentimelineio/composable.h
    Adds the bounds method to the public interface
    Python Bindings: Also added

  • src/opentimelineio/composition.cpp
    Implements the has_clips method for composition, which returns true if the Composition contains at least one clip. This function is used by bounds method of the derived Stack class to exclude any tracks that do not contain any clips when it calculates the union of the bounds. Since the bounds default to width, height and center of {0, 0, (0, 0)} we want to prevent the default values from affecting the result and thus use the has_clips method to skip over any tracks that do not contain clips.

  • src/opentimelineio/composition.h
    Adds the has_clips method to the public interface.
    Python Bindings: Also added
    Note: Even though this method is only needed by Stack, I thought it could be useful elsewhere so I made it public. If you would prefer I could make it protected so only Stack could access it. Or if you do not want it in the interface at all, I can also implement it as a lambda function (or in the anonymous namespace) inside of Stack::bounds.

  • src/opentimelineio/deserialization.cpp
    Implements the de/serialization of Point, Box and optional primitives. The code follows the existing standard provided by Time and TimeRange.

  • src/opentimelineio/errorStatus.cpp
    Adds a new error value for when the bounds cannot be determined. This occurs when the user asks for bounds on a Clip that does not contain bounds or asks for bounds on a Composition that has a Clip that does not contain bounds.

  • src/opentimelineio/externalReference.cpp, src/opentimelineio/generatorReference.cpp, src/opentimelineio/imageSequenceReference.cpp, src/opentimelineio/missingReference.h
    Pass bounds constructor argument to MediaReference parent constructor

  • src/opentimelineio/externalReference.h, src/opentimelineio/generatorReference.h, src/opentimelineio/imageSequenceReference.h, src/opentimelineio/missingReference.h
    Adds optional constructor argument for bounds.

  • src/opentimelineio/mediaReference.cpp
    Initializes bounds member and add it to the read/write method for serialization.

  • src/opentimelineio/mediaReference.h
    Add bounds member and constructor argument as well as a mutator and accessor method.
    Python Bindings: Also added (includes derived classes)

  • src/opentimelineio/safely_typed_any.cpp
    Adds casting support for Box and Point (required by serialization code).

  • src/opentimelineio/safely_typed_any.h
    Adds casting methods for Box and Point to public interface.

  • src/opentimelineio/serializableObject.h
    Adds de/serialization support for Box and Point found within the MediaReference schema.

  • src/opentimelineio/serialization.cpp
    Adds de/serialization code for Box and Point.

  • src/opentimelineio/stack.cpp
    Implements bounds method for a stack. We begin by finding the first bounding box, defined as the Box of the first Clip or of the first Composition that contains at least one Clip. After this is found, we then calculate the union of this with each subsequent Clip or Composition containing at least one Clip. If no Clips are found, we returning the default Box. If an error is found, we return the error (as a parameter) and the default Box. This follows the standard defined by the available_range method.

  • src/opentimelineio/stack.h
    Adds bounds method to the public interface.
    Python Bindings: Also added

  • src/opentimelineio/timeline.h
    Adds bounds to the public interface and implements it by redirecting the call to it's Stack.
    Python Bindings: Also added

  • src/opentimelineio/track.cpp
    Implements bounds method for a track. Similarly to Stack, we begin by finding the first bounding box, however in this case we don't need to consider Compositions, only Clips. Once we have the bounding Box of the first clip, we get the union of it with the Box of each subsequent Clip. If no Clips are found, we returning the default Box. If an error is found, we return the error (as a parameter) and the default Box. This follows the standard defined by the available_range method.

  • src/opentimelineio/track.h
    Adds bounds method to the public interface.
    Python Bindings: Also added

  • src/opentimelineio/version.h
    Adds Point and Box to the versioned opentimelineio namespace.

  • src/py-opentimelineio/opentime-bindings/CMakeLists.txt
    Adds compilation of Python bindings for Box and Point.

  • src/py-opentimelineio/opentime-bindings/opentime_bindings.cpp
    Call Point and Box Python bindings.

  • src/py-opentimelineio/opentime-bindings/opentime_bindings.h
    Adds prototypes for Point and Box bindings as well as prototypes for the Python str and repr functions for Points (since this is reused by Box str and repr methods).

  • src/py-opentimelineio/opentime-bindings/opentime_box.cpp
    Adds implementation of Python bindings for the Box class
    Note: These are denoted by Python Bindings in Box.h description above.

  • src/py-opentimelineio/opentime-bindings/opentime_point.cpp
    Adds implementation of Python bindings for the Point class
    Note: These are denoted by Python Bindings in Point.h description above.

  • src/py-opentimelineio/opentimelineio-bindings/otio_serializableObjects.cpp
    Adds all added opentimelineio public c++ interface methods to the Python bindings
    Note: These are denoted by Python Bindings in descriptions above.

  • src/py-opentimelineio/opentimelineio/core/mediaReference.py
    Adds bounds to the repr and str method for MediaReference.

  • src/py-opentimelineio/opentimelineio/opentime.py
    Adds Point and Box to the opentime module.

  • src/py-opentimelineio/opentimelineio/schema/image_sequence_reference.py
    Adds bounds to the repr and str method for ImageSequence.

Tests

  • tests/baselines/*.json
    Adds empty bounds to the json representations
    Note: This is necessary even if the bounds don't exist in the metadata since the serialization code will add a null bounds member in this case. If you'd prefer I could make it exclude the bounds member entirely if it is not defined.

  • tests/sample_data/clip_example.otio
    Added bounds to the example clip and associated test (test_documentation.py).

  • tests/test_clip.py
    Updated the str test to include the bounds. Also adds a bounding box to a clip and tests that they were set correctly using the contains methods to find the edges of the Box.

  • tests/test_composition.py
    Tests the added has_clips method on empty stacks and tracks and ones containing no Clips (only Gaps). Also tests that the bounds are correctly applied to empty Stacks, single clip Stacks, multi-clip Stacks, Stacks with gaps, and Stacks containing Tracks containing Clips. Tracks are also tested for empty Tracks, gap-only Tracks, single Clip Tracks and multi-Clip Tracks.

  • tests/test_documentation.py
    Deserializes the updated clip_example.otio file and make sure it contains the bounds.

  • tests/test_generator_reference.py, tests/test_image_sequence_reference.py, tests/test_media_reference.py
    Ensures the bounds can be passed to the constructor and the same bounds are returned from the accessor.
    ImageSequenceReference and MediaRefererence also test the str and repr methods return the expected string.

  • tests/test_opentime.py
    Adds Point and Box tests to test all the publicly available basic methods: equality, inequality, accessor, and mutators, and well as the Python copy, deepcopy, str and repr. The Box test also adds tests for aspect_ratio and aspect_ratio with 0 width. The contains method tests Boxes centered at the the default (0,0) and a non-default value. The union method is tests in three situation: completely overlapping boxes, partially overlapping boxes and non-overlapping boxes.

  • tests/test_timeline.py
    Timeline bounds are tested to ensure a single track timeline has equal bounds between the Track and the Timeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

MediaReference: spatial bounding box
1 participant