Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop passing classes and default_classes to export and evaluation methods #1858

Merged
merged 2 commits into from
Jun 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/source/integrations/coco.rst
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,8 @@ to add them to your dataset as follows:
]

# Add COCO predictions to `predictions` field of dataset
fouc.add_coco_labels(coco_dataset, "predictions", predictions)
classes = coco_dataset.default_classes
fouc.add_coco_labels(coco_dataset, "predictions", predictions, classes)

# Verify that predictions were added to two images
print(coco_dataset.count("predictions")) # 2
Expand Down
17 changes: 4 additions & 13 deletions docs/source/integrations/cvat.rst
Original file line number Diff line number Diff line change
Expand Up @@ -431,9 +431,8 @@ details:
`label_field` or all fields in `label_schema` without classes specified.
All new label fields must have a class list provided via one of the
supported methods. For existing label fields, if classes are not provided
by this argument nor `label_schema`, they are retrieved from
:meth:`Dataset.get_classes() <fiftyone.core.dataset.Dataset.get_classes>`
if possible, or else the observed labels on your dataset are used
by this argument nor `label_schema`, the observed labels on your dataset
are used
- **attributes** (*True*): specifies the label attributes of each label field
to include (other than their `label`, which is always included) in the
annotation export. Can be any of the following:
Expand Down Expand Up @@ -631,16 +630,8 @@ FiftyOne can infer the appropriate values to use:

- **label_type**: if omitted, the |Label| type of the field will be used to
infer the appropriate value for this parameter
- **classes**: if omitted for a non-semantic segmentation field, the class
lists from the :meth:`classes <fiftyone.core.dataset.Dataset.classes>` or
:meth:`default_classes <fiftyone.core.dataset.Dataset.default_classes>`
properties of your dataset will be used, if available. Otherwise, the
observed labels on your dataset will be used to construct a classes list
- **mask_targets**: if omitted for a semantic segmentation field, the mask
targets from the
:meth:`mask_targets <fiftyone.core.dataset.Dataset.mask_targets>` or
:meth:`default_mask_targets <fiftyone.core.dataset.Dataset.default_mask_targets>`
properties of your dataset will be used, if available
- **classes**: if omitted for a non-semantic segmentation field, the observed
labels on your dataset will be used to construct a classes list

.. _cvat-label-attributes:

Expand Down
17 changes: 4 additions & 13 deletions docs/source/user_guide/annotation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -472,9 +472,8 @@ more details:
`label_field` or all fields in `label_schema` without classes specified.
All new label fields must have a class list provided via one of the
supported methods. For existing label fields, if classes are not provided
by this argument nor `label_schema`, they are retrieved from
:meth:`Dataset.get_classes() <fiftyone.core.dataset.Dataset.get_classes>`
if possible, or else the observed labels on your dataset are used
by this argument nor `label_schema`, the observed labels on your dataset
are used
- **attributes** (*True*): specifies the label attributes of each label field
to include (other than their `label`, which is always included) in the
annotation export. Can be any of the following:
Expand Down Expand Up @@ -644,16 +643,8 @@ FiftyOne can infer the appropriate values to use:

- **label_type**: if omitted, the |Label| type of the field will be used to
infer the appropriate value for this parameter
- **classes**: if omitted for a non-semantic segmentation field, the class
lists from the :meth:`classes <fiftyone.core.dataset.Dataset.classes>` or
:meth:`default_classes <fiftyone.core.dataset.Dataset.default_classes>`
properties of your dataset will be used, if available. Otherwise, the
observed labels on your dataset will be used to construct a classes list
- **mask_targets**: if omitted for a semantic segmentation field, the mask
targets from the
:meth:`mask_targets <fiftyone.core.dataset.Dataset.mask_targets>` or
:meth:`default_mask_targets <fiftyone.core.dataset.Dataset.default_mask_targets>`
properties of your dataset will be used, if available
- **classes**: if omitted for a non-semantic segmentation field, the observed
labels on your dataset will be used to construct a classes list

.. _annotation-label-attributes:

Expand Down
18 changes: 9 additions & 9 deletions docs/source/user_guide/dataset_creation/datasets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1553,16 +1553,16 @@ COCO format:
dataset.export(
export_dir="/tmp/coco",
dataset_type=fo.types.COCODetectionDataset,
classes=classes,
label_field="ground_truth",
classes=classes,
)

# Export predictions
dataset.export(
dataset_type=fo.types.COCODetectionDataset,
labels_path="/tmp/coco/predictions.json",
classes=classes,
label_field="predictions",
classes=classes,
)

# Now load ground truth labels into a new dataset
Expand All @@ -1577,7 +1577,7 @@ COCO format:
dataset2,
"predictions",
"/tmp/coco/predictions.json",
classes=classes,
classes,
)

# Verify that ground truth and predictions were imported as expected
Expand Down Expand Up @@ -2147,16 +2147,16 @@ images-and-labels and labels-only data in YOLO format:
dataset.export(
export_dir="/tmp/yolov4",
dataset_type=fo.types.YOLOv4Dataset,
classes=classes,
label_field="ground_truth",
classes=classes,
)

# Export predictions
dataset.export(
dataset_type=fo.types.YOLOv4Dataset,
labels_path="/tmp/yolov4/predictions",
classes=classes,
label_field="predictions",
classes=classes,
)

# Now load ground truth labels into a new dataset
Expand All @@ -2171,7 +2171,7 @@ images-and-labels and labels-only data in YOLO format:
dataset2,
"predictions",
"/tmp/yolov4/predictions",
classes=classes,
classes,
)

# Verify that ground truth and predictions were imported as expected
Expand Down Expand Up @@ -2327,16 +2327,16 @@ images-and-labels and labels-only data in YOLO format:
export_dir="/tmp/yolov5",
dataset_type=fo.types.YOLOv5Dataset,
split="validation",
classes=classes,
label_field="ground_truth",
classes=classes,
)

# Export predictions
view.export(
dataset_type=fo.types.YOLOv5Dataset,
labels_path="/tmp/yolov5/predictions/validation",
classes=classes,
label_field="predictions",
classes=classes,
)

# Now load ground truth labels into a new dataset
Expand All @@ -2352,7 +2352,7 @@ images-and-labels and labels-only data in YOLO format:
dataset2,
"predictions",
"/tmp/yolov5/predictions/validation",
classes=classes,
classes,
)

# Verify that ground truth and predictions were imported as expected
Expand Down
12 changes: 6 additions & 6 deletions docs/source/user_guide/dataset_creation/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -369,16 +369,16 @@ predictions to take advantage of FiftyOne's
dataset.export(
export_dir="/tmp/coco",
dataset_type=fo.types.COCODetectionDataset,
classes=classes,
label_field="ground_truth",
classes=classes,
)

# Export predictions
dataset.export(
dataset_type=fo.types.COCODetectionDataset,
labels_path="/tmp/coco/predictions.json",
classes=classes,
label_field="predictions",
classes=classes,
)

# Now load ground truth labels into a new dataset
Expand All @@ -393,7 +393,7 @@ predictions to take advantage of FiftyOne's
dataset2,
"predictions",
"/tmp/coco/predictions.json",
classes=classes,
classes,
)

# Verify that ground truth and predictions were imported as expected
Expand Down Expand Up @@ -432,16 +432,16 @@ predictions to take advantage of FiftyOne's
dataset.export(
export_dir="/tmp/yolov4",
dataset_type=fo.types.YOLOv4Dataset,
classes=classes,
label_field="ground_truth",
classes=classes,
)

# Export predictions
dataset.export(
dataset_type=fo.types.YOLOv4Dataset,
labels_path="/tmp/yolov4/predictions",
classes=classes,
label_field="predictions",
classes=classes,
)

# Now load ground truth labels into a new dataset
Expand All @@ -456,7 +456,7 @@ predictions to take advantage of FiftyOne's
dataset2,
"predictions",
"/tmp/yolov4/predictions",
classes=classes,
classes,
)

# Verify that ground truth and predictions were imported as expected
Expand Down
37 changes: 9 additions & 28 deletions docs/source/user_guide/export_datasets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -349,26 +349,16 @@ Class lists
-----------

Certain labeled image/video export formats such as
:ref:`COCO <COCODetectionDataset-export>` or
:ref:`COCO <COCODetectionDataset-export>` and
:ref:`YOLO <YOLOv5Dataset-export>` store an explicit list of classes for the
label field being exported.

In such cases, all exporters provided by FiftyOne use the following strategy
to retrieve the class list when exporting a given `label_field`:
By convention, all exporters provided by FiftyOne should provide a `classes`
parameter that allows for manually specifying the classes list to use.

1. If the exporter provides a parameter like `classes` that allows for
manually specifying the classes, this list is used
2. If the :meth:`classes <fiftyone.core.dataset.Dataset.classes>` dict of the
sample collection contains `label_field`, this class list is used
3. If the collection's
:meth:`default_classes <fiftyone.core.dataset.Dataset.default_classes>`
attribute is non-empty, this class list is used
4. If the collection's :meth:`info <fiftyone.core.dataset.Dataset.info>` dict
contains a class list under the `classes` key, this list is used

If no explicit class list is available via the above methods, the observed
classes in the collection being exported are used, which may be a subset of the
classes in the parent dataset when exporting a view.
If no explicit class list is provided, the observed classes in the collection
being exported are used, which may be a subset of the classes in the parent
dataset when exporting a view.

.. note::

Expand Down Expand Up @@ -397,26 +387,17 @@ classes in the parent dataset when exporting a view.
# Create a view that only contains cats and dogs
view = dataset.filter_labels("ground_truth", F("label").is_in(["cat", "dog"]))

# By default, `default_classes` will be used to populate the COCO categories
# By default, only the observed classes will be stored as COCO categories
view.export(
labels_path="/tmp/coco1.json",
dataset_type=fo.types.COCODetectionDataset,
)

# But, since we're only interested in exporting cats and dogs, we can override
# this by manually providing the `classes` argument
# However, if desired, we can explicitly provide a classes list
view.export(
labels_path="/tmp/coco2.json",
dataset_type=fo.types.COCODetectionDataset,
classes=["cat", "dog"],
)

# Equivalently, we can clear `default_classes` first so that the observed
# labels (only cats and dogs in this case) will be used
dataset.default_classes = None
view.export(
labels_path="/tmp/coco3.json",
dataset_type=fo.types.COCODetectionDataset,
classes=dataset.default_classes,
)

.. _supported-export-formats:
Expand Down
17 changes: 3 additions & 14 deletions fiftyone/core/collections.py
Original file line number Diff line number Diff line change
Expand Up @@ -2010,11 +2010,7 @@ def evaluate_classifications(
instances
eval_key (None): a string key to use to refer to this evaluation
classes (None): the list of possible classes. If not provided,
classes are loaded from
:meth:`fiftyone.core.dataset.Dataset.classes` or
:meth:`fiftyone.core.dataset.Dataset.default_classes` if
possible, or else the observed ground truth/predicted labels
are used
the observed ground truth/predicted labels are used
missing (None): a missing label string. Any None-valued labels
are given this label for results purposes
method ("simple"): a string specifying the evaluation method to use.
Expand Down Expand Up @@ -2116,11 +2112,7 @@ def evaluate_detections(
or :class:`fiftyone.core.labels.TemporalDetections`
eval_key (None): a string key to use to refer to this evaluation
classes (None): the list of possible classes. If not provided,
classes are loaded from
:meth:`fiftyone.core.dataset.Dataset.classes` or
:meth:`fiftyone.core.dataset.Dataset.default_classes` if
possible, or else the observed ground truth/predicted labels
are used
the observed ground truth/predicted labels are used
missing (None): a missing label string. Any unmatched objects are
given this label for results purposes
method (None): a string specifying the evaluation method to use.
Expand Down Expand Up @@ -2211,10 +2203,7 @@ def evaluate_segmentations(
instances
eval_key (None): a string key to use to refer to this evaluation
mask_targets (None): a dict mapping mask values to labels. If not
provided, mask targets are loaded from
:meth:`fiftyone.core.dataset.Dataset.mask_targets` or
:meth:`fiftyone.core.dataset.Dataset.default_mask_targets` if
possible, or else the observed pixel values are used
provided, the observed pixel values are used
method ("simple"): a string specifying the evaluation method to
use. Supported values are ``("simple")``
**kwargs: optional keyword arguments for the constructor of the
Expand Down
16 changes: 2 additions & 14 deletions fiftyone/utils/annotations.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,10 +115,8 @@ def annotate(
``label_field`` or all fields in ``label_schema`` without classes
specified. All new label fields must have a class list provided via
one of the supported methods. For existing label fields, if classes
are not provided by this argument nor ``label_schema``, they are
parsed from
:meth:`fiftyone.core.collections.SampleCollection.get_classes` if
possible, or else the observed labels on your dataset are used
are not provided by this argument nor ``label_schema``, the
observed labels on your dataset are used
attributes (True): specifies the label attributes of each label
field to include (other than their ``label``, which is always
included) in the annotation export. Can be any of the
Expand Down Expand Up @@ -772,10 +770,6 @@ def _get_classes(
% label_field
)

classes = samples.get_classes(label_field)
if classes:
return classes

_, label_path = samples._get_label_field_path(label_field, "label")
return sorted(
set(samples._dataset.distinct(label_path))
Expand Down Expand Up @@ -815,12 +809,6 @@ def _get_mask_targets(samples, mask_targets, label_field, label_info):
if "mask_targets" in label_info:
mask_targets = label_info["mask_targets"]

if mask_targets is None and label_field in samples.mask_targets:
mask_targets = samples.mask_targets[label_field]

if mask_targets is None and samples.default_mask_targets:
mask_targets = samples.default_mask_targets

if mask_targets is None:
mask_targets = {i: str(i) for i in range(1, 256)}
mask_targets[0] = "background"
Expand Down
Loading