Merge back 1.4.0 to develop (#2387)

* Temporarily skip visual prompting openvino integration test (#2323)

* Fix import dm.DatasetSubset (#2324)

* Fix semantic segmentation soft prediction dtype (#2322)

* Contrain yapf verison lesser than 0.40.0 (#2328)

* Fix detection e2e tests (#2327)

* Mergeback: Label addtion/deletion 1.2.4 --> 1.4.0 (#2326)

* OTX deploy for visual prompting task  (#2311)

* Bump albumentations version in anomaly requirements (#2350)

* Update action detection (#2346)

* Add OTX optimize for visual prompting task (#2318)

* Update detection docs (#2335)

* Add visual prompting documentation (#2354)

* Remove custom modelapi patch in visual prompting (#2359)

* Fix graph metric order and label issues (#2356)

* Update multi-label document and conversion script (#2358)

* Update third party programs (#2365)

* Make anomaly task compatible with older albumentations versions (#2363)

* Fixing detection saliency map for one class case (#2368)

* Add e2e test for visual prompting (#2360)

* Fix e2e (#2366)

* Add Dino head unit tests (#2344)

* Update for release 1.4.0rc2 (#2370)

* Fix NNCF training on CPU (#2373)

* Align label order between Geti and OTX (#2369)

* Remove CenterCrop from Classification test pipeline and editing missing docs link (#2375)

* Fix H-label classification (#2377)

* Update for release 1.4 (#2380)

* Switch to PTQ for sseg (#2374)

* Fix invalid import structures in otx.api (#2383)

* Update for 1.4.0rc4 (#2385)

---------

Signed-off-by: Kim, Vinnam <[email protected]>
Co-authored-by: Kim, Sungchul <[email protected]>
Co-authored-by: Vinnam Kim <[email protected]>
Co-authored-by: Evgeny Tsykunov <[email protected]>
Co-authored-by: Songki Choi <[email protected]>
Co-authored-by: Eunwoo Shin <[email protected]>
Co-authored-by: Jaeguk Hyun <[email protected]>
Co-authored-by: Sungman Cho <[email protected]>
Co-authored-by: Eugene Liu <[email protected]>
Co-authored-by: Wonju Lee <[email protected]>
Co-authored-by: Dick Ameln <[email protected]>
Co-authored-by: Vladislav Sovrasov <[email protected]>
Co-authored-by: sungchul.kim <sungchul@ikvensx010>
Co-authored-by: GalyaZalesskaya <[email protected]>
Co-authored-by: Harim Kang <[email protected]>

.github/workflows/daily.yml

@@ -24,6 +24,8 @@ jobs:
             test_dir: "tests/e2e/cli/instance_segmentation"
           - task: "seg"
             test_dir: "tests/e2e/cli/semantic_segmentation"
+          - task: "visprompt"
+            test_dir: "tests/e2e/cli/visual_prompting"
     name: E2E-Test-py310-${{ matrix.task }}
     uses: ./.github/workflows/run_tests_in_tox.yml
     with:

.github/workflows/pre_merge.yml

@@ -104,6 +104,8 @@ jobs:
             test_dir: "tests/integration/cli/action"
           - task: "ano"
             test_dir: "tests/integration/cli/anomaly"
+          - task: "visprompt"
+            test_dir: "tests/integration/cli/visual_prompting"
     name: Integration-Test-py310-${{ matrix.task }}
     # This is what will cancel the job concurrency
     concurrency:

CHANGELOG.md

@@ -26,27 +26,33 @@ All notable changes to this project will be documented in this file.
 
 - Support encrypted dataset training (<https://github.com/openvinotoolkit/training_extensions/pull/2209>)
 - Add custom max iou assigner to prevent CPU OOM when large annotations are used (<https://github.com/openvinotoolkit/training_extensions/pull/2228>)
-- Auto train type detection for Semi-SL, Self-SL and Incremental: "--train-type" now is optional (https://github.com/openvinotoolkit/training_extensions/pull/2195)
-- Add per-class XAI saliency maps for Mask R-CNN model (https://github.com/openvinotoolkit/training_extensions/pull/2227)
+- Auto train type detection for Semi-SL, Self-SL and Incremental: "--train-type" now is optional (<https://github.com/openvinotoolkit/training_extensions/pull/2195>)
+- Add per-class XAI saliency maps for Mask R-CNN model (<https://github.com/openvinotoolkit/training_extensions/pull/2227>)
 - Add new object detector Deformable DETR (<https://github.com/openvinotoolkit/training_extensions/pull/2249>)
-- Add new object detector DINO(<https://github.com/openvinotoolkit/training_extensions/pull/2266>)
-- Add new visual prompting task (https://github.com/openvinotoolkit/training_extensions/pull/2203), (https://github.com/openvinotoolkit/training_extensions/pull/2274)
+- Add new object detector DINO (<https://github.com/openvinotoolkit/training_extensions/pull/2266>)
+- Add new visual prompting task (<https://github.com/openvinotoolkit/training_extensions/pull/2203>, <https://github.com/openvinotoolkit/training_extensions/pull/2274>, <https://github.com/openvinotoolkit/training_extensions/pull/2311>, <https://github.com/openvinotoolkit/training_extensions/pull/2354>, <https://github.com/openvinotoolkit/training_extensions/pull/2318>)
 - Add new object detector ResNeXt101-ATSS (<https://github.com/openvinotoolkit/training_extensions/pull/2309>)
 
 ### Enhancements
 
 - Introduce channel_last parameter to improve the performance (<https://github.com/openvinotoolkit/training_extensions/pull/2205>)
-- Decrease a time for making a workspace (<https://github.com/openvinotoolkit/training_extensions/pull/2223>)
+- Decrease time for making a workspace (<https://github.com/openvinotoolkit/training_extensions/pull/2223>)
 - Set persistent_workers and pin_memory as True in detection task (<https://github.com/openvinotoolkit/training_extensions/pull/2224>)
-- New algorithm for Semi-SL semantic segmentation based on metric lerning via class prototypes (https://github.com/openvinotoolkit/training_extensions/pull/2156)
-- Self-SL for classification now can recieve just folder with any images to start contrastive pretraining (https://github.com/openvinotoolkit/training_extensions/pull/2219)
+- New algorithm for Semi-SL semantic segmentation based on metric learning via class prototypes (<https://github.com/openvinotoolkit/training_extensions/pull/2156>)
+- Self-SL for classification now can recieve just folder with any images to start contrastive pretraining (<https://github.com/openvinotoolkit/training_extensions/pull/2219>)
 - Update OpenVINO version to 2023.0, and NNCF verion to 2.5 (<https://github.com/openvinotoolkit/training_extensions/pull/2090>)
-- Improve XAI saliency map generation for tiling detection and tiling instance segmentation (https://github.com/openvinotoolkit/training_extensions/pull/2240)
+- Improve XAI saliency map generation for tiling detection and tiling instance segmentation (<https://github.com/openvinotoolkit/training_extensions/pull/2240>)
+- Remove CenterCrop from Classification test pipeline and editing missing docs link(<https://github.com/openvinotoolkit/training_extensions/pull/2375>)
+- Switch to PTQ for sseg (<https://github.com/openvinotoolkit/training_extensions/pull/2374>)
 
 ### Bug fixes
 
 - Fix the bug that auto adapt batch size is unavailable with IterBasedRunner (<https://github.com/openvinotoolkit/training_extensions/pull/2182>)
 - Fix the bug that learning rate isn't scaled when multi-GPU trianing is enabled(<https://github.com/openvinotoolkit/training_extensions/pull/2254>)
+- Fix the bug that label order is misaligned when model is deployed from Geti (<https://github.com/openvinotoolkit/training_extensions/pull/2369>)
+- Fix NNCF training on CPU (<https://github.com/openvinotoolkit/training_extensions/pull/2373>)
+- Fix H-label classification (<https://github.com/openvinotoolkit/training_extensions/pull/2377>)
+- Fix invalid import structures in otx.api (<https://github.com/openvinotoolkit/training_extensions/pull/2383>)
 
 ### Known issues
 

docs/source/guide/explanation/algorithms/classification/multi_label_classification.rst

@@ -24,7 +24,7 @@ Dataset Format
 **************
 
 As it is a common practice to use object detection datasets in the academic area, we support the most popular object detection format: `COCO <https://cocodataset.org/#format-data>`_.
-Specifically, this format should be converted in our `internal representation <https://github.com/openvinotoolkit/training_extensions/tree/develop/data/datumaro_multilabel>`_ first. We provided a `script <https://github.com/openvinotoolkit/training_extensions/blob/develop/src/otx/algorithms/classification/utils/convert_coco_to_multilabel.py>` to help with conversion.
+Specifically, this format should be converted in our `internal representation <https://github.com/openvinotoolkit/training_extensions/tree/develop/tests/assets/datumaro_multilabel>`_ first. We provided a `script <https://github.com/openvinotoolkit/training_extensions/blob/develop/src/otx/algorithms/classification/utils/convert_coco_to_multilabel.py>` to help with conversion.
 To convert the COCO data format to our internal one, run this script in similar way:
 
 .. code-block::
@@ -35,6 +35,9 @@ To convert the COCO data format to our internal one, run this script in similar
 
     Please, refer to our :doc:`dedicated tutorial <../../../tutorials/base/how_to_train/classification>` for more information how to train, validate and optimize classification models.
 
+.. note::
+    For now, "___" is a symbol to distinguish the multi-label format. So, it must be included at the front of the label name. 
+
 ******
 Models
 ******

docs/source/guide/explanation/algorithms/index.rst

@@ -31,3 +31,4 @@ Contents
    segmentation/index
    anomaly/index
    action/index
+   visual_prompting/index