Add point and box geometry primivites and use them to create MediaReference bounding boxes #773
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #771
MediaReference: spatial bounding box
Summary
This change addresses the need to have spatial bounds expressed in the MediaReference. These are used to define a measurement-agnostic coordinate system representing the viewing area of the clip(s), which can be used by media players to determine how to display or scale/resize/center (if necessary) the media referenced by the Clips.
For a more in-depth explanation of the idea and motivations please see the issue linked here:
#771
I have flagged as many potential discussion points that I think might arise with Note below.
Note
Files changed
*.md files
Lists the additions to the schemas (automatically generated by Makefiles).
src/opentime/CMakeLists.txt
Adds new Point and Box files to the compilation
src/opentime/box.h
Adds a new Box primitive with members width, height and center, including accessor and mutators for each. The default constructor, operator== and operator!= required by the serialization code is also added. Lastly, some common convenience methods are added: get_aspect_ratio, contains and get_union.
The get_aspect_ratio method returns simply the ratio of width over height, commonly used by media applications. The contains method returns true if Point argument is contained within the Box. Lastly, the get_union method take a Box and returns the minimum sized box required to contain both Boxes. This is used by Composition/Timeline to calculate the union of all the Clips contained within them.
Note: The contains and get_aspect_ratio methods are really only used by the unit tests, but I thought they were useful. I can remove it if you prefer not to have it.
Note: To prevent a divide by 0, get_aspect_ratio returns 1 if the width is 0. If you'd prefer another default value, let me know.
Python Bindings: Also added for all public functions.
src/opentime/point.h
Adds a new Point primitive with members x and y, including accessor and mutators for each. The default constructor, operator== and operator!= required by the serialization code is also added.
The class is mainly used by Box to define it's center.
Note: I added a convenience templated is_equal method for floating point values here, that is also used by Box. If you'd prefer it to go somewhere else (like a utility file), let me know.
Python Bindings: Also added for all public functions.
src/opentimelineio/clip.cpp
Implement the bounds method. Simply returns the bounding box if it has been set on the clip or an error otherwise (conforming to the standard of the available_range method).
src/opentimelineio/clip.h
Adds the bounds method to the public interface.
Python Bindings: Also added
src/opentimelineio/composable.cpp
Implements the bounds method to return an error so that any dervied classes that do not explicitly override the virtual method return the error (it is overriden by Clip, Stack and Track).
src/opentimelineio/composable.h
Adds the bounds method to the public interface
Python Bindings: Also added
src/opentimelineio/composition.cpp
Implements the has_clips method for composition, which returns true if the Composition contains at least one clip. This function is used by bounds method of the derived Stack class to exclude any tracks that do not contain any clips when it calculates the union of the bounds. Since the bounds default to width, height and center of {0, 0, (0, 0)} we want to prevent the default values from affecting the result and thus use the has_clips method to skip over any tracks that do not contain clips.
src/opentimelineio/composition.h
Adds the has_clips method to the public interface.
Python Bindings: Also added
Note: Even though this method is only needed by Stack, I thought it could be useful elsewhere so I made it public. If you would prefer I could make it protected so only Stack could access it. Or if you do not want it in the interface at all, I can also implement it as a lambda function (or in the anonymous namespace) inside of Stack::bounds.
src/opentimelineio/deserialization.cpp
Implements the de/serialization of Point, Box and optional primitives. The code follows the existing standard provided by Time and TimeRange.
src/opentimelineio/errorStatus.cpp
Adds a new error value for when the bounds cannot be determined. This occurs when the user asks for bounds on a Clip that does not contain bounds or asks for bounds on a Composition that has a Clip that does not contain bounds.
src/opentimelineio/externalReference.cpp, src/opentimelineio/generatorReference.cpp, src/opentimelineio/imageSequenceReference.cpp, src/opentimelineio/missingReference.h
Pass bounds constructor argument to MediaReference parent constructor
src/opentimelineio/externalReference.h, src/opentimelineio/generatorReference.h, src/opentimelineio/imageSequenceReference.h, src/opentimelineio/missingReference.h
Adds optional constructor argument for bounds.
src/opentimelineio/mediaReference.cpp
Initializes bounds member and add it to the read/write method for serialization.
src/opentimelineio/mediaReference.h
Add bounds member and constructor argument as well as a mutator and accessor method.
Python Bindings: Also added (includes derived classes)
src/opentimelineio/safely_typed_any.cpp
Adds casting support for Box and Point (required by serialization code).
src/opentimelineio/safely_typed_any.h
Adds casting methods for Box and Point to public interface.
src/opentimelineio/serializableObject.h
Adds de/serialization support for Box and Point found within the MediaReference schema.
src/opentimelineio/serialization.cpp
Adds de/serialization code for Box and Point.
src/opentimelineio/stack.cpp
Implements bounds method for a stack. We begin by finding the first bounding box, defined as the Box of the first Clip or of the first Composition that contains at least one Clip. After this is found, we then calculate the union of this with each subsequent Clip or Composition containing at least one Clip. If no Clips are found, we returning the default Box. If an error is found, we return the error (as a parameter) and the default Box. This follows the standard defined by the available_range method.
src/opentimelineio/stack.h
Adds bounds method to the public interface.
Python Bindings: Also added
src/opentimelineio/timeline.h
Adds bounds to the public interface and implements it by redirecting the call to it's Stack.
Python Bindings: Also added
src/opentimelineio/track.cpp
Implements bounds method for a track. Similarly to Stack, we begin by finding the first bounding box, however in this case we don't need to consider Compositions, only Clips. Once we have the bounding Box of the first clip, we get the union of it with the Box of each subsequent Clip. If no Clips are found, we returning the default Box. If an error is found, we return the error (as a parameter) and the default Box. This follows the standard defined by the available_range method.
src/opentimelineio/track.h
Adds bounds method to the public interface.
Python Bindings: Also added
src/opentimelineio/version.h
Adds Point and Box to the versioned opentimelineio namespace.
src/py-opentimelineio/opentime-bindings/CMakeLists.txt
Adds compilation of Python bindings for Box and Point.
src/py-opentimelineio/opentime-bindings/opentime_bindings.cpp
Call Point and Box Python bindings.
src/py-opentimelineio/opentime-bindings/opentime_bindings.h
Adds prototypes for Point and Box bindings as well as prototypes for the Python str and repr functions for Points (since this is reused by Box str and repr methods).
src/py-opentimelineio/opentime-bindings/opentime_box.cpp
Adds implementation of Python bindings for the Box class
Note: These are denoted by Python Bindings in Box.h description above.
src/py-opentimelineio/opentime-bindings/opentime_point.cpp
Adds implementation of Python bindings for the Point class
Note: These are denoted by Python Bindings in Point.h description above.
src/py-opentimelineio/opentimelineio-bindings/otio_serializableObjects.cpp
Adds all added opentimelineio public c++ interface methods to the Python bindings
Note: These are denoted by Python Bindings in descriptions above.
src/py-opentimelineio/opentimelineio/core/mediaReference.py
Adds bounds to the repr and str method for MediaReference.
src/py-opentimelineio/opentimelineio/opentime.py
Adds Point and Box to the opentime module.
src/py-opentimelineio/opentimelineio/schema/image_sequence_reference.py
Adds bounds to the repr and str method for ImageSequence.
Tests
tests/baselines/*.json
Adds empty bounds to the json representations
Note: This is necessary even if the bounds don't exist in the metadata since the serialization code will add a null bounds member in this case. If you'd prefer I could make it exclude the bounds member entirely if it is not defined.
tests/sample_data/clip_example.otio
Added bounds to the example clip and associated test (test_documentation.py).
tests/test_clip.py
Updated the str test to include the bounds. Also adds a bounding box to a clip and tests that they were set correctly using the contains methods to find the edges of the Box.
tests/test_composition.py
Tests the added has_clips method on empty stacks and tracks and ones containing no Clips (only Gaps). Also tests that the bounds are correctly applied to empty Stacks, single clip Stacks, multi-clip Stacks, Stacks with gaps, and Stacks containing Tracks containing Clips. Tracks are also tested for empty Tracks, gap-only Tracks, single Clip Tracks and multi-Clip Tracks.
tests/test_documentation.py
Deserializes the updated clip_example.otio file and make sure it contains the bounds.
tests/test_generator_reference.py, tests/test_image_sequence_reference.py, tests/test_media_reference.py
Ensures the bounds can be passed to the constructor and the same bounds are returned from the accessor.
ImageSequenceReference and MediaRefererence also test the str and repr methods return the expected string.
tests/test_opentime.py
Adds Point and Box tests to test all the publicly available basic methods: equality, inequality, accessor, and mutators, and well as the Python copy, deepcopy, str and repr. The Box test also adds tests for aspect_ratio and aspect_ratio with 0 width. The contains method tests Boxes centered at the the default (0,0) and a non-default value. The union method is tests in three situation: completely overlapping boxes, partially overlapping boxes and non-overlapping boxes.
tests/test_timeline.py
Timeline bounds are tested to ensure a single track timeline has equal bounds between the Track and the Timeline.