Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CVAT support for frame_start, frame_stop, and frame_step #3181

Merged
merged 10 commits into from
Jun 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions docs/source/integrations/cvat.rst
Original file line number Diff line number Diff line change
Expand Up @@ -505,6 +505,34 @@ provided:
or when using `task_size` and generating multiple tasks
- **organization** (*None*): the name of the organization to use when sending
requests to CVAT
- **frame_start** (*None*): nonnegative integer(s) defining the first frame
of videos to upload when creating video tasks. Supported values are:

- `integer`: the first frame to upload for each video
- `list`: a list of first frame integers corresponding to videos in the
given samples
- `dict`: a dictionary mapping sample filepaths to first frame integers
to use for the corresponding videos

- **frame_stop** (*None*): nonnegative integer(s) defining the last frame of
videos to upload when creating video tasks. Supported values are:

- `integer`: the last frame to upload for each video
- `list`: a list of last frame integers corresponding to videos in the
given samples
- `dict`: a dictionary mapping sample filepaths to last frame integers to
use for the corresponding videos

- **frame_step** (*None*): positive integer(s) defining which frames to
sample when creating video tasks. Supported values are:

- `integer`: the frame step to apply to each video task
- `list`: a list of frame step integers corresponding to videos in the
given samples
- `dict`: a dictionary mapping sample filepaths to frame step integers to
use for the corresponding videos

Note that this argument cannot be provided when uploading existing tracks

.. _cvat-label-schema:

Expand Down Expand Up @@ -1030,6 +1058,8 @@ supported values are:
- `"prompt"` (**default**): present an interactive prompt to direct/discard
unexpected labels
- `"ignore"`: automatically ignore any unexpected labels
- ``"keep"``: automatically keep all unexpected labels in a field whose name
matches the the label type
- `"return"`: return a dict containing all unexpected labels, if any

See :ref:`this section <cvat-unexpected-annotations>` for more details.
Expand Down Expand Up @@ -1482,6 +1512,8 @@ supported values are:

- `"prompt"` (**default**): present an interactive prompt to direct/discard
unexpected labels
- ``"keep"``: automatically keep all unexpected labels in a field whose name
matches the the label type
- `"ignore"`: automatically ignore any unexpected labels
- `"return"`: return a dict containing all unexpected labels, if any

Expand Down Expand Up @@ -2126,6 +2158,66 @@ destination fields.
)
dataset.delete_annotation_run(anno_key)

.. _cvat-frame-args:

Using frame start, stop, step
-----------------------------

When annotating videos, you can use the arguments `frame_start`, `frame_stop`,
and `frame_step` to annotate subsampled clips of your videos rather than
loading every frame into CVAT. These arguments are only supported for video
tasks and accept either integer values to use for each video task that is
created, a list of values that will be applied to video tasks in a round-robin
strategy, or a dictionary of values mapping the video filepath to the
corresponding integer value.

Note: Uploading existing annotation tracks while using the `frame_step`
argument is not currently supported.

.. code:: python
:linenos:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-video", max_samples=2).clone()
sample_fps = dataset.values("filepath")

# Start video 1 at frame 10 and video 2 at frame 5
frame_start = {sample_fps[0]: 10, sample_fps[1]: 5}

# For video 1, load every frame after the start
# For video 2, load every 10th frame
frame_step = [1, 10]

# Stop all videos at frame 100
frame_stop = 100

anno_key = "frame_args"
label_field = "frames.new_detections"
label_type = "detections"
classes = ["person", "vehicle"]

# Annotate a new detections field
dataset.annotate(
anno_key,
label_field=label_field,
label_type=label_type,
classes=classes,
frame_start=frame_start,
frame_stop=frame_stop,
frame_step=frame_step,
)
print(dataset.get_annotation_info(anno_key))

# Annotate in CVAT

dataset.load_annotations(
anno_key,
cleanup=True,
)
dataset.delete_annotation_run(anno_key)

.. _cvat-annotating-videos:

Annotating videos
Expand Down
2 changes: 2 additions & 0 deletions docs/source/user_guide/annotation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -927,6 +927,8 @@ supported values are:

- `"prompt"` (**default**): present an interactive prompt to direct/discard
unexpected labels
- ``"keep"``: automatically keep all unexpected labels in a field whose name
matches the the label type
- `"ignore"`: automatically ignore any unexpected labels
- `"return"`: return a dict containing all unexpected labels, if any

Expand Down
2 changes: 2 additions & 0 deletions fiftyone/core/collections.py
Original file line number Diff line number Diff line change
Expand Up @@ -8363,6 +8363,8 @@ def load_annotations(
- ``"prompt"``: present an interactive prompt to
direct/discard unexpected labels
- ``"ignore"``: automatically ignore any unexpected labels
- ``"keep"``: automatically keep all unexpected labels in a
field whose name matches the the label type
- ``"return"``: return a dict containing all unexpected
labels, or ``None`` if there aren't any
cleanup (False): whether to delete any informtation regarding this
Expand Down
Loading