[FR] Allow patches exports to pull labels from Detection attributes/fields

[geoffrp]

Proposal Summary

After using FiftyOne and CVAT to organize my datasets, I've encountered a relatively common pattern of exporting patches into different classification datasets for secondary analysis. For instance, if my object detector was training on detecting apples, I may assign an attribute in CVAT to describe the color of a selected apple. To ease the requirements on my object detector, I would implement a second classifier to perform the attribute classification downstream. Right now, exporting patches seems to be limited to exporting images with labels assigned to Detections, not attributes stored in fields or the attributes dictionary of said label classes.

I think the export method should define an interface to make this type of exporting possible.

Motivation

• What is the use case for this feature?

Users can now export image classification datasets from attributes rather than the original object detection labels, adding support for a two-step detect then classify pipeline.

• Why is this use case valuable to support for FiftyOne users in general?

It seems to be a common pattern for designers who wish to constrain the complexity of their object detector's labeling schema and instead pass it along to a second stage classifier.

• Why is this use case valuable to support for your project(s) or
  organization?

Several of my projects utilize this pattern and I don't see that changing too soon.

• Why is it currently difficult to achieve this use case? (please be as
  specific as possible about why related FiftyOne features and components are
  insufficient)

Custom scripting isn't necessarily hard to write for this case, but allowing annotations to carry attributes meant for image classification applications seems to suggest this feature be in place.

What areas of FiftyOne does this feature affect?

• App: FiftyOne application
• Core: Core fiftyone Python library
• Server: FiftyOne server

Details

It seems like the best options would be to add additional configuration to the ImagePatchesExtractor via the export method. Maybe something like label_schema for CVAT annotations that allows users to specify an attribute or multiple attributes to assign to each extracted patch?

There may be good reasons to also implement a custom transforms argument for ImagePatchesExtractor that would allow for greater control over the final look of extracted patches than the original alpha and force_square. This could be argued to be irrelevant as at training time users can apply whatever transforms they wish to properly format images to a model's input.

Willingness to contribute

The FiftyOne Community encourages new feature contributions. Would you or
another member of your organization be willing to contribute an implementation
of this feature?

• Yes. I can contribute this feature independently.
• Yes. I would be willing to contribute this feature with guidance from
  the FiftyOne community.
• No. I cannot contribute this feature at this time.

[geoffrp]

I'll add that I think both image classification and multi-label classification should be supported here.

[brimoor]

Hey @geoffrp! LMK if #1993 covers this issue 😄

[geoffrp]

@brimoor Late to the approve here - yes, thank you! Much appreciated.