Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable DINO to OTX - Step 1. Enable Deformable DETR to OTX #2249

Merged
merged 6 commits into from
Jun 22, 2023
Merged

Enable DINO to OTX - Step 1. Enable Deformable DETR to OTX #2249

merged 6 commits into from
Jun 22, 2023

Conversation

jaegukhyun
Copy link
Contributor

@jaegukhyun jaegukhyun commented Jun 16, 2023 •
edited
Loading

Summary

As a first step to bring DINO to OTX, this PR enable Deformable DETR to OTX.
Since mmdeploy do not support Deformable DETR(it supports only DETR), there are some custom patches for mmcv and mmdeploy ops

Known Issue

  1. Current Deformable DETR use fixed schedule, not early stopping and plateau lr decay
  2. Current Deformable DETR does not support smart weight loading, and other class incremental technique
  3. Current DetClassProbabilityMapHook does not support End-to-End detector
  4. When inference with exported Deformable DETR, there are some bottleneck for some images
    image

These known issues are handled after implementing DINO from this PR

How to test

Unit tests and integration tests are added. E2E Test will be added after implementing DINO

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@github-actions github-actions bot added ALGO Any changes in OTX Algo Tasks implementation DOC Improvements or additions to documentation TEST Any changes in tests labels Jun 16, 2023
@jaegukhyun jaegukhyun marked this pull request as ready for review June 19, 2023 02:08
@jaegukhyun jaegukhyun requested a review from a team as a code owner June 19, 2023 02:08
eunwoosh
eunwoosh reviewed Jun 19, 2023
View reviewed changes
Copy link
Contributor

@eunwoosh eunwoosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding great model :)
I left some comments. please take a look.

otx/algorithms/common/adapters/mmcv/ops/multi_scale_deformable_attn_pytorch.py Outdated Show resolved Hide resolved
otx/algorithms/common/adapters/mmcv/ops/multi_scale_deformable_attn_pytorch.py Outdated Show resolved Hide resolved
otx/algorithms/detection/adapters/mmdet/models/detectors/custom_deformable_detr_detector.py Outdated Show resolved Hide resolved
otx/algorithms/detection/configs/detection/resnet50_deformable-detr/data_pipeline.py Show resolved Hide resolved
otx/algorithms/detection/configs/detection/resnet50_deformable-detr/template_experimental.yaml Show resolved Hide resolved
otx/algorithms/detection/configs/detection/resnet50_deformable-detr/template_experimental.yaml Show resolved Hide resolved
otx/algorithms/detection/configs/detection/resnet50_deformable-detr/template_experimental.yaml Show resolved Hide resolved
tests/unit/algorithms/common/adapters/mmdeploy/ops/test_custom_ops.py Outdated Show resolved Hide resolved
harimkang
harimkang reviewed Jun 19, 2023
View reviewed changes
Copy link
Contributor

@harimkang harimkang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to the new algorithm. It looks good for the most part, and I have a few comments.

otx/algorithms/common/adapters/mmcv/ops/multi_scale_deformable_attn_pytorch.py Show resolved Hide resolved
otx/algorithms/common/adapters/mmdeploy/ops/custom_ops.py Show resolved Hide resolved
otx/algorithms/detection/adapters/mmdet/models/detectors/custom_deformable_detr_detector.py Show resolved Hide resolved
@jaegukhyun jaegukhyun added this to the 1.4.0 milestone Jun 21, 2023
harimkang
harimkang approved these changes Jun 22, 2023
View reviewed changes
Copy link
Contributor

@harimkang harimkang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

BTW, Anything to add to the documentation for this?

@jaegukhyun
Copy link
Contributor Author

LGTM!

BTW, Anything to add to the documentation for this?

Documentation will be updated after adding DINO and conducting some experiments on public datasets

@jaegukhyun jaegukhyun merged commit cbfffbe into openvinotoolkit:develop Jun 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eunwoosh eunwoosh eunwoosh approved these changes

@sungmanc sungmanc sungmanc approved these changes

@harimkang harimkang harimkang approved these changes

Assignees
No one assigned
Labels
ALGO Any changes in OTX Algo Tasks implementation DOC Improvements or additions to documentation TEST Any changes in tests
Projects
None yet
Milestone
1.4.0
Development

Successfully merging this pull request may close these issues.
4 participants
@jaegukhyun @harimkang @sungmanc @eunwoosh