Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes in exportable code #2031

Merged
merged 12 commits into from
Apr 20, 2023
Merged

Fixes in exportable code #2031

merged 12 commits into from
Apr 20, 2023

Conversation

GalyaZalesskaya
Copy link
Contributor

@GalyaZalesskaya GalyaZalesskaya commented Apr 18, 2023 •
edited
Loading

Summary

Fixes in exportable code:

  • Fixed torch imports in otx/algorithms/common/utils/utils.py, so the demo in exportable code doesn't require torch to run anymore
  • Removed the condition that ACTION_CLASSIFICATION and ACTION_DETECTION TaskType and Domain appear in enum only if FEATURE_FLAGS_OTX_ACTION_TASKS flag is used. This caused an issue that enum initialization happened before initializing FEATURE_FLAGS_OTX_ACTION_TASKS as 1 for CLI tasks here, so the demo failed with attribute error for ACTION CLASSIFICATION.

Enhances:

  • Removed coping __pycache__ to deployed zip from exportable code
  • Moved utility function for saving the demo output to otx.api.utils to avoid cross-dependencies on otx.cli in exportable code package

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@github-actions github-actions bot added ALGO Any changes in OTX Algo Tasks implementation API Any changes in OTX API CLI Any changes in OTE CLI TEST Any changes in tests labels Apr 18, 2023
@GalyaZalesskaya GalyaZalesskaya added BUG Something isn't working RELEASE and removed CLI Any changes in OTE CLI ALGO Any changes in OTX Algo Tasks implementation labels Apr 18, 2023
@github-actions github-actions bot added ALGO Any changes in OTX Algo Tasks implementation CLI Any changes in OTE CLI and removed TEST Any changes in tests labels Apr 18, 2023
@GalyaZalesskaya GalyaZalesskaya marked this pull request as ready for review April 18, 2023 22:05
@GalyaZalesskaya GalyaZalesskaya requested a review from a team as a code owner April 18, 2023 22:05
sungmanc
sungmanc reviewed Apr 18, 2023
View reviewed changes
Copy link
Contributor

@sungmanc sungmanc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments.

otx/api/entities/label.py Outdated Show resolved Hide resolved
otx/api/entities/model_template.py Outdated Show resolved Hide resolved
goodsong81
goodsong81 requested changes Apr 19, 2023
View reviewed changes
Copy link
Contributor

@goodsong81 goodsong81 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACTION_CLASSIFICATION/DETECTION should not be exposed to Geti yet, it's the reason why we adopted the FEATURE_FLAG mechanism. Can you fix the issue in other ways?

If you mean the exportable code demo.py, why don't you add os.environ["FEATURE_FLAGS_OTX_ACTION_TASKS"] = "1" in the demo.py before importing otx?

@GalyaZalesskaya
Copy link
Contributor Author

@sungmanc @goodsong81 Thank you for the clarification about ACTION_CLASSIFICATION/DETECTION flag, I fixed its usage

@goodsong81 goodsong81 merged commit 5b3116c into openvinotoolkit:releases/1.2.0 Apr 20, 2023
goodsong81 added a commit that referenced this pull request Apr 21, 2023
@goodsong81 @harimkang
@eugene123tw @sungmanc @yunchu @jaegukhyun @cih9088 @eunwoosh @supersoob @GalyaZalesskaya @vinnamkim
Merge back 1.2.0rc2 (#2045)
d80a8bd 
* HOT-FIX: Revert segmentation model's ignore mode in CLI (#2011)

Revert segmentation ignore=True

* Improve tiling preprocess (#2013)

* prevent timeout during init phase

* Fix reg tests (#2008)

* Edit regression tests

* Change the dataset root

* Miss typo

* Fix pre-commit

* Fix openvino import error due to Tiler init import (#2015)

Remove init import for Tiler to prevent OpenVINO import

* Bump up version to 1.2.0 (#2017)

* Set the python version to "3.10" for code-scan workflow

* Add missing __init__.py (#2019)

* Add missing __init__.py

* Change license

* Release 1.2.0rc1

* Fix issue that str2bool not being applied in certain cases (#2023)

* Add workaround solution

* Fix minor

* Remove str int

* Fix default dict (#2025)

fix: change default to configdict

Signed-off-by: Inhyuk Andy Cho <[email protected]>

* Convert dummy datasets to toy datasets (#1988)

* Update cls, det datsets

* Remove useless files

* Change action datasets

* Edit action dataset

* change dir

* Add xml files

* Remove useless

* Edite tets

* Fix tests

* Fix tests

* Remove ptc

* Remove

* Fix precommit

* Update dataset, fix cls bug

* Remove useless dataset

* Edit drop_last

* Fix missed part

* Change threshold values to unifying

* bugfix: squeezing to 1 dimenetion

* Change threshold for deployment

* Fix multi-gpu issue, e2e tests

* Decrease num_workers for tiling test and tiling processes

* Revert num_workers for tests

* Fix datsets

---------

Co-authored-by: eunwoosh <[email protected]>

* Fix E2E tests (#2032)

* Optimize data preprocessing time and enhance overall performance in semantic segmentation (#2020)

* HOTFIX: change doc version to 1.2.0

* Add storage cache in Apache Arrow format using Datumaro (#2009)

* feat: change label entity to dictionay

* feat: add datumaro arrow cache

* refacor: move to proper directory

* fix: align to the latest

* fix: align data to otx

* fix: align new version

* refactor: disable storage cache for action tasks

* test: fix

* fix: version back

* docs: add to changelog

* fix: keep __height, __width

* docs: add description

* test: revert tests

* fix: revert back to list

* style: ruff

* HOT-FIX: Revert segmentation model's ignore mode in CLI (Develop) (#2012)

Revert segmentation ignore=True

* fix: make force verbose

* test: add storage cache test

* feat: datumaro 1.2.0

* test: test path exists

* test: do deepcopy

* style: make black happy

---------

Signed-off-by: Inhyuk Andy Cho <[email protected]>
Co-authored-by: Harim Kang <[email protected]>

* Fix typo in prediction_to_annotation_converter.py (#2028)

* HOT-FIX: Revert segmentation model's ignore mode in CLI (Develop) (#2012)

Revert segmentation ignore=True

* Bump up version to 1.3.0rc0 (#2016)

* bug fix

* del

* revert

* revert changlog

---------

Co-authored-by: Harim Kang <[email protected]>
Co-authored-by: Songki Choi <[email protected]>

* Fix tiling config loading bug (#2030)

* fix tiling loading bug

* fix mypy

* Make multi gpu child processes done right after evaluation (#2033)

* Fixes in exportable code (#2031)

* Create Actions domain and task type despite flag

* Move import torch to the function

* Fix str in dump_features

* Move dump_frames to otx/api/utils

* Remove __pycache__ from dunped exportable code

* Add tests for demo --output option

* Update sha for exportable code requirements

* Add flag to task_type_to_label_domain

* Roll back tests creation to add it in separate PR

* Add FEATURE_FLAGS_OTX_ACTION_TASKS init in demo.py

* Remove extra comments

* Fix linter

* Add documentation for the noisy label detection feature (#2034)

* Add documentation for noisy label detection feature

Signed-off-by: Kim, Vinnam <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Kim, Vinnam <[email protected]>

* Add documentation

Signed-off-by: Kim, Vinnam <[email protected]>

* Fix typo

Signed-off-by: Kim, Vinnam <[email protected]>

* Fix small typo

---------

Signed-off-by: Kim, Vinnam <[email protected]>
Co-authored-by: Songki Choi <[email protected]>

* Remove skip tests, fix regression tests (#2036)

* Remove skip tests, fix regression tests

* Fix precommit

* Hide internal options from external GUI (#2037)

Signed-off-by: Songki Choi <[email protected]>

* Add unit test for classification task and configurer (#2035)

* Reduce the depth of aumix process (#2038)

Enable light augmix

* Change samples_per_gpu in _infer_model(Detection) (#2041)

Change samples_per_gpu in _infer_model

* Decrease batch size if CUDA OOM occurs (#2022)

* implement adpating bs

* refine impl

* implement adaptive bs also in cls, seg task

* refine adapt bs algo to consider gpu util

* refactor code

* write comment and docstring

* implement decreasig bs on action task

* update learning rate after decreasing batch size

* implement test code of mmcv automatic_bs file

* remove meta modification

* remove unused improt

* implement test code of torch automatic_bs file

* align with pre commit

* add line to tell not supporting anomaly

* update CHANGELOG

* update docs

* change argument help

* change file name

* apply pr comment

* add auto_decrease_bs in learning parameters

* align with pre commit

* fix typo

* add integration test

* bugfix

* update test code

* not execute algo in nncf

* suppor nncf

* apply comment

* align with pre commit

* change method to set value

* refine warning comment

* remove breakpoint

* make hpo not use auto decrease batch size

* refine warning & typo fix

* align with pre commit

* Update CHANGELOG.md

Co-authored-by: Sungman Cho <[email protected]>

* update unit test

* update integration test

* bufix

* Release 1.2.0rc2

Signed-off-by: Songki Choi <[email protected]>

* Update OTX commit hash for exportable code requiements

---------

Signed-off-by: Inhyuk Andy Cho <[email protected]>
Signed-off-by: Kim, Vinnam <[email protected]>
Signed-off-by: Songki Choi <[email protected]>
Co-authored-by: Harim Kang <[email protected]>
Co-authored-by: Eugene Liu <[email protected]>
Co-authored-by: Sungman Cho <[email protected]>
Co-authored-by: Yunchu Lee <[email protected]>
Co-authored-by: Jaeguk Hyun <[email protected]>
Co-authored-by: Inhyuk Cho <[email protected]>
Co-authored-by: eunwoosh <[email protected]>
Co-authored-by: Soobee Lee <[email protected]>
Co-authored-by: Galina Zalesskaya <[email protected]>
Co-authored-by: Vinnam Kim <[email protected]>
@goodsong81 goodsong81 mentioned this pull request Apr 24, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sungmanc sungmanc sungmanc left review comments

@sovrasov sovrasov sovrasov left review comments

@goodsong81 goodsong81 goodsong81 approved these changes

Assignees
No one assigned
Labels
ALGO Any changes in OTX Algo Tasks implementation API Any changes in OTX API BUG Something isn't working CLI Any changes in OTE CLI RELEASE
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@GalyaZalesskaya @goodsong81 @sovrasov @sungmanc