All notable changes to this project will be documented in this file.
- Add YOLOv9 model for Object Detection (openvinotoolkit#3917)
- Add OV inference for keypoint detection (openvinotoolkit#3970)
- Add tiling for semantic segmentation (openvinotoolkit#3954)
- Add 3D Object Detection task with MonoDETR model (openvinotoolkit#3979)
- Upgrade OV, MAPI, and NNCF dependencies (openvinotoolkit#3967)
- Instance Segmentation Model refactoring (openvinotoolkit#3865)
- Bump torch and lightning to 2.4.0 versions (openvinotoolkit#3843)
- Add mAP metric to evaluate multilabel classification (openvinotoolkit#3985)
- Fix a wrong HPO log (openvinotoolkit#3972)
- Add RT-DETR model for Object Detection (openvinotoolkit#3741)
- Add Multi-Label & H-label Classification with torchvision models (openvinotoolkit#3697)
- Add Hugging-Face Model Wrapper for Classification (openvinotoolkit#3710)
- Add LoRA finetuning capability for ViT Architectures (openvinotoolkit#3729)
- Add Hugging-Face Model Wrapper for Object Detection (openvinotoolkit#3747)
- Add Hugging-Face Model Wrapper for Semantic Segmentation (openvinotoolkit#3749)
- Enable torch.compile to work with classification (openvinotoolkit#3758)
- Add
otx benchmark
subcommand (openvinotoolkit#3762) - Add RTMPose for Keypoint Detection Task (openvinotoolkit#3781)
- Add Semi-SL MeanTeacher algorithm for Semantic Segmentation (openvinotoolkit#3801)
- Update head and h-label format for hierarchical label classification (openvinotoolkit#3810)
- Support configurable input size (openvinotoolkit#3788)
- Reimplement of ViT Architecture following TIMM (openvinotoolkit#3719)
- Enable to override data configurations (openvinotoolkit#3748)
- Enable to use input_size at transforms in recipe (openvinotoolkit#3759)
- Enable to use polygon and bitmap mask as prompt inputs for zero-shot learning (openvinotoolkit#3769)
- Refactoring
ConvModule
by removingconv_cfg
,norm_cfg
, andact_cfg
(openvinotoolkit#3783, openvinotoolkit#3816, openvinotoolkit#3809) - Support ImageFromBytes (openvinotoolkit#3948)
- Enable model export (openvinotoolkit#3952)
- Move templates from OTX1.X to OTX2.X (openvinotoolkit#3951)
- Include Geti arrow dataset subset names (openvinotoolkit#3962)
- Include full image with anno in case there's no tile in tile dataset (openvinotoolkit#3964)
- Add type checker in converter for callable functions (optimizer, scheduler) (openvinotoolkit#3968)
- Change sematic segmentation to consider bbox only annotations (openvinotoolkit#3996)
- Fix Combined Dataloader & unlabeled warmup loss in Semi-SL (openvinotoolkit#3723)
- Revert #3579 to fix issues with replacing coco_instance with a different format in some dataset (openvinotoolkit#3753)
- Add num_devices in Engine for multi-gpu training (openvinotoolkit#3778)
- Add missing tile recipes and various tile recipe changes (openvinotoolkit#3942)
- Change categories mapping logic (openvinotoolkit#3946)
- Fix config converter for tiling (openvinotoolkit#3973)
NOTES
OpenVINO™ Training Extensions, version 2.1.0 does not include the latest functional and security updates. OpenVINO™ Training Extensions, version 2.2.0 is targeted to be released in September 2024 and will include additional functional and security updates. Customers should update to the latest version as it becomes available.
- Add a flag to enable OV inference on dGPU (openvinotoolkit#3503)
- Add early stopping with warmup. Remove mandatory background label in semantic segmentation task (openvinotoolkit#3515)
- RTMDet-tiny enablement for detection task (openvinotoolkit#3542)
- Add data_format validation and update in OTXDataModule (openvinotoolkit#3579)
- Add torchvision.MaskRCNN (openvinotoolkit#3504)
- Add Semi-SL for Multi-class Classification (EfficientNet-B0) (openvinotoolkit#3566)
- Decoupling mmaction for action classification (MoviNet, X3D) (openvinotoolkit#3582)
- Add Semi-SL Algorithms for mv3-large, effnet-v2, deit-tiny, dino-v2 (openvinotoolkit#3602)
- RTMDet-tiny enablement for detection task (export/optimize) (openvinotoolkit#3564)
- Enable ruff & ruff-format into otx/algo/classification/backbones (openvinotoolkit#3667)
- Add TV MaskRCNN Tile Recipe (openvinotoolkit#3655)
- Add rotated det OV recipe (openvinotoolkit#3687)
- Change load_stat_dict to on_load_checkpoint (openvinotoolkit#3443)
- Add try - except to keep running the remaining tests (openvinotoolkit#3448)
- Update instance_segmentation.py to resolve conflict with 2.0.0 (openvinotoolkit#3506)
- Update XPU install (openvinotoolkit#3516)
- Sync rgb order between torch and ov inference of action classification task (openvinotoolkit#3551)
- Make Perf test available to load pervious Perf test to skip training stage (openvinotoolkit#3556)
- Reenable e2e classification XAI tests (openvinotoolkit#3591)
- Remove action detection task support (openvinotoolkit#3605)
- Increase readability of pickling error log during HPO & fix minor bug (openvinotoolkit#3606)
- Update RTMDet checkpoint url (openvinotoolkit#3631)
- Refactor Torchvision Model for Classification Semi-SL (openvinotoolkit#3614)
- Add coverage omit mm-related code (openvinotoolkit#3641)
- Add docs semi-sl part (openvinotoolkit#3640)
- Refactor docs design & Add contents (openvinotoolkit#3645)
- Add execution example of auto batch size in docs (openvinotoolkit#3648)
- Add Semi-SL for cls Benchmark Test (openvinotoolkit#3647)
- Move value to device before logging for metric (openvinotoolkit#3649)
- Add .codecov.yaml (openvinotoolkit#3650)
- Update benchmark tool for otx2.1 (openvinotoolkit#3652)
- Collect pretrained weight binary files in one place (openvinotoolkit#3656)
- Minimize compiled dependency files (openvinotoolkit#3653)
- Update README & CODEOWNERS (openvinotoolkit#3659)
- Update Engine's docstring & CLI --help outputs (openvinotoolkit#3658)
- Align integration test to exportable code interface update for release branch (openvinotoolkit#3676)
- Refactor exporter for anomaly task and fix a bug with exportable code (openvinotoolkit#3672)
- Update pandas version constraint (openvinotoolkit#3679)
- Include more models to export test into test_otx_e2e (openvinotoolkit#3678)
- Move assigning tasks to Models from Engine to Anomaly Model Classes (openvinotoolkit#3683)
- Refactoring detection modules (openvinotoolkit#3636)
- Fix conflicts between develop and 2.0.0 (openvinotoolkit#3490)
- Fix polygon mask (openvinotoolkit#3549)
- Fix vpm intg test error (openvinotoolkit#3554)
- Fix anomaly (openvinotoolkit#3557)
- Bug fix in Semantic Segmentation + enable DINOV2 export in ONNX (openvinotoolkit#3569)
- Fix some export issues. Remove EXPORTABLE_CODE as export parameter. (openvinotoolkit#3577)
- Fix
load_from_checkpoint
to apply original model's hparams (openvinotoolkit#3607) - Fix
load_from_checkpoint
args to apply original model's hparams (openvinotoolkit#3611) - Fix zero-shot
learn
for ov model (openvinotoolkit#3601) - Various fixes for XAI in 2.1 (openvinotoolkit#3615)
- Fix tests to work in a mm-free environment (openvinotoolkit#3632)
- Fix a bug in benchmark code (openvinotoolkit#3643)
- Update exportable code dependency & fix a bug (openvinotoolkit#3642)
- Fix getting wrong shape during resizing (openvinotoolkit#3644)
- Fix detection prediction outputs (openvinotoolkit#3634)
- Fix RTMDet PTQ performance (openvinotoolkit#3651)
- Fix segmentation fault on VPM PTQ (openvinotoolkit#3654, openvinotoolkit#3689)
- Fix NNCF MaskRCNN-Eff accuracy drop (openvinotoolkit#3680)
- Fix optimize with Semi-SL data pipeline (openvinotoolkit#3684)
- Fix MaskRCNN SwinT NNCF Accuracy Drop (openvinotoolkit#3685)
- Fix MaskRCNN SwinT NNCF Accuracy Drop By Adding More PTQ Configs (openvinotoolkit#3929)
- Post-Training Quantization (PTQ) optimization applied to maskrcnn_swint in the instance segmentation task may result in significantly reduced accuracy. This issue is expected to be addressed with an upgrade to OpenVINO and NNCF in a future release.
NOTES
OpenVINO™ Training Extensions which version 2.0.0 has been updated to include refactoring of the overall architecture and functional updates. Users should install the new environment.
- Enable New design to provide a more seamless API/CLI that delivers the value of OTX: Product Design
- Moved away from MMLab's libraries to provide a Lightning-based core and training pipeline
- Use Lightning-based modules and trainers to deliver APIs/CLIs in a more user-friendly way
- Support Intel devices for accelerating deep learning model training
- Support more models for each task
- Improve the API so user can configure efficient training with shorter code
- Provide more customize settings through the CLI and API
- Enhance the Auto-Configuration feature and made it available in the API
- Fixing some minor issues
- Anomaly task processing times have increased compared with v1.* version, with anomaly classification experiencing a slowdown of approximately 26%, anomaly detection by approximately 213%, and anomaly segmentation by approximately 78%. Issue #3592
- Post-Training Quantization (PTQ) optimization applied to
maskrcnn_swint
in the instance segmentation task may result in significantly reduced accuracy compared with v1.* Issue #3593
- Update pymongo version to 4.6.3 for resolving CVE-2024-21506 (openvinotoolkit#3396)
- Use torchvision in MRCNN on CUDA (openvinotoolkit#3347)
- Update IPEX version in installation guide documentation (openvinotoolkit#3343)
- Update benchmark (openvinotoolkit#3338)
- Bump idan version to 3.7 (openvinotoolkit#3332)
- Support benchmark history summary (openvinotoolkit#3307)
- Pin pymongo version to 4.5.0 (openvinotoolkit#3316)
- Upgrade MAPI (openvinotoolkit#3304)
- Add NMS iou threshold configurable parameter (openvinotoolkit#3287)
- Remedy some medium/low severity bandit issues (openvinotoolkit#3208)
- Update documentations (openvinotoolkit#3280)
- Add perf benchmark test cases for action and visual prompting (openvinotoolkit#3292)
- Explicitly cast incorrect output type in OV model (openvinotoolkit#3395)
- Update QAT configs for rotated detection (openvinotoolkit#3375)
- Hotfix 🔧 Bypass ClsIncrSampler for tiling (openvinotoolkit#3374)
- [NNCF] Dynamic shape datasets WA (openvinotoolkit#3355)
- [Hotfix] 🔥 Fixing detection oriented OV inferencer (openvinotoolkit#3351)
- Revert adaptive batch size (openvinotoolkit#3340)
- Fix e2e tests for XPU (openvinotoolkit#3305)
- Remove torch.xpu.optimize for semantic_segmentation task (openvinotoolkit#3172)
- Changed supported Python version range (>=3.9, <=3.11) (openvinotoolkit/datumaro#1269)
- Support MMDetection COCO format (openvinotoolkit/datumaro#1213)
- Develop JsonSectionPageMapper in Rust API (openvinotoolkit/datumaro#1224)
- Add Filtering via User-Provided Python Functions (openvinotoolkit/datumaro#1230, openvinotoolkit/datumaro#1233)
- Remove supporting MacOS platform (openvinotoolkit/datumaro#1235)
- Support Kaggle image data (
KaggleImageCsvBase
,KaggleImageTxtBase
,KaggleImageMaskBase
,KaggleVocBase
,KaggleYoloBase
) (openvinotoolkit/datumaro#1240) - Add
__getitem__()
for random accessing with O(1) time complexity (openvinotoolkit/datumaro#1247) - Add Data-aware Anchor Generator (openvinotoolkit/datumaro#1251)
- Support bounding box import within Kaggle extractors and add
KaggleCocoBase
(openvinotoolkit/datumaro#1273)
- Optimize Python import to make CLI entrypoint faster (openvinotoolkit/datumaro#1182)
- Add ImageColorScale context manager (openvinotoolkit/datumaro#1194)
- Enhance visualizer to toggle plot title visibility (openvinotoolkit/datumaro#1228)
- Enhance Datumaro data format detect() to be memory-bounded and performant (openvinotoolkit/datumaro#1229)
- Change RoIImage and MosaicImage to have np.uint8 dtype as default (openvinotoolkit/datumaro#1245)
- Enable image backend and color channel format to be selectable (openvinotoolkit/datumaro#1246)
- Boost up
CityscapesBase
andKaggleImageMaskBase
by droppingnp.unique
(openvinotoolkit/datumaro#1261) - Enhance RISE algortihm for explainable AI (openvinotoolkit/datumaro#1263)
- Enhance explore unit test to use real dataset from ImageNet (openvinotoolkit/datumaro#1266)
- Fix each method of the comparator to be used separately (openvinotoolkit/datumaro#1290)
- Bump ONNX version to 1.16.0 (openvinotoolkit/datumaro#1376)
- Print the color channel format (RGB) for datum stats command (openvinotoolkit/datumaro#1389)
- Add ignore_index argument to Mask.as_class_mask() and Mask.as_instance_mask() (openvinotoolkit/datumaro#1409)
- Fix wrong example of Datumaro dataset creation in document (openvinotoolkit/datumaro#1195)
- Fix wrong command to install datumaro from github (openvinotoolkit/datumaro#1202, openvinotoolkit/datumaro#1207)
- Update document to correct wrong
datum project import
command and add filtering example to filter out items containing annotations. (openvinotoolkit/datumaro#1210) - Fix label compare of distance method (openvinotoolkit/datumaro#1205)
- Fix Datumaro visualizer's import errors after introducing lazy import (openvinotoolkit/datumaro#1220)
- Fix broken link to supported formats in readme (openvinotoolkit/datumaro#1221)
- Fix Kinetics data format to have media data (openvinotoolkit/datumaro#1223)
- Handling undefined labels at the annotation statistics (openvinotoolkit/datumaro#1232)
- Add unit test for item rename (openvinotoolkit/datumaro#1237)
- Fix a bug in the previous behavior when importing nested datasets in the project (openvinotoolkit/datumaro#1243)
- Fix Kaggle importer when adding duplicated labels (openvinotoolkit/datumaro#1244)
- Fix input tensor shape in model interpreter for OpenVINO 2023.3 (openvinotoolkit/datumaro#1251)
- Add default value for target in prune cli (openvinotoolkit/datumaro#1253)
- Remove deprecated MediaManager (openvinotoolkit/datumaro#1262)
- Fix explore command without project (openvinotoolkit/datumaro#1271)
- Fix enable COCO to import only bboxes (openvinotoolkit/datumaro#1360)
- Fix resize transform for RleMask annotation
- (openvinotoolkit/datumaro#1361)
- Fix import YOLO variants from extractor when
urls
is not specified (openvinotoolkit/datumaro#1362)
- Add memory bounded datumaro data format detect to release 1.5.1 (openvinotoolkit/datumaro#1241)
- Bump version string to 1.5.2 (openvinotoolkit/datumaro#1249)
- Remove Protobuf version limitation (<4) (openvinotoolkit/datumaro#1248)
- Enhance Datumaro data format stream importer performance (openvinotoolkit/datumaro#1153)
- Change image default dtype from float32 to uint8 (openvinotoolkit/datumaro#1175)
- Add comparison level-up doc (openvinotoolkit/datumaro#1174)
- Add ImportError to catch GitPython import error (openvinotoolkit/datumaro#1174)
- Modify the draw function in the visualizer not to raise an error for unsupported annotation types. (openvinotoolkit/datumaro#1180)
- Correct explore path in the related document. (openvinotoolkit/datumaro#1176)
- Fix errata in the voc document. Color values in the labelmap.txt should be separated by commas, not colons. (openvinotoolkit/datumaro#1162)
- Fix hyperlink errors in the document (openvinotoolkit/datumaro#1159, openvinotoolkit/datumaro#1161)
- Fix memory unbounded Arrow data format export/import (openvinotoolkit/datumaro#1169)
- Update CVAT format doc to bypass warning (openvinotoolkit/datumaro#1183)
- Enable configurable confidence threshold for otx eval and export (openvinotoolkit#2388)
- Add YOLOX variants as new object detector models (openvinotoolkit#2402)
- Enable FeatureVectorHook to support action tasks (openvinotoolkit#2408)
- Add ONNX metadata to detection, instance segmentation, and segmentation models (openvinotoolkit#2418)
- Add a new feature to configure input size (openvinotoolkit#2420)
- Introduce the OTXSampler and AdaptiveRepeatDataHook to achieve faster training at the small data regime (openvinotoolkit#2428)
- Add a new object detector Lite-DINO (openvinotoolkit#2457)
- Add Semi-SL Mean Teacher algorithm for Instance Segmentation task (openvinotoolkit#2444)
- Official supports for YOLOX-X, YOLOX-L, YOLOX-S, ResNeXt101-ATSS (openvinotoolkit#2485)
- Add new argument to track resource usage in train command (openvinotoolkit#2500)
- Add Self-SL for semantic segmentation of SegNext families (openvinotoolkit#2215)
- Adapt input size automatically based on dataset statistics (openvinotoolkit#2499)
- Refine input data in-memory caching (openvinotoolkit#2416)
- Adapt timeout value of initialization for distributed training (openvinotoolkit#2422)
- Optimize data loading by merging load & resize operations w/ caching support for cls/det/iseg/sseg (openvinotoolkit#2438, openvinotoolkit#2453, openvinotoolkit#2460)
- Support torch==2.0.1 (openvinotoolkit#2465)
- Set "Auto" as default input size mode (openvinotoolkit#2515)
- Fix F1 auto-threshold to choose best largest confidence (openvinotoolkit#2371)
- Fix IBLoss enablement with DeiT-Tiny when class incremental training (openvinotoolkit#2594)
- OpenVINO(==2023.0) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch>=1.13.1
- NNCF QAT optimization is disabled for MaskRCNN models due to CUDA runtime error in ROIAlign kernel on torch==2.0.1
- Update ModelAPI configuration(openvinotoolkit#2564)
- Add Anomaly modelAPI changes (openvinotoolkit#2563)
- Update Image numpy access (openvinotoolkit#2586)
- Make max_num_detections configurable (openvinotoolkit#2647)
- Fix IBLoss enablement with DeiT-Tiny when class incremental training (openvinotoolkit#2595)
- Fix mmcls bug not wrapping model in DataParallel on CPUs (openvinotoolkit#2601)
- Fix h-label loss normalization issue w/ exclusive label group of singe label (openvinotoolkit#2604)
- Fix division by zero in class incremental learning for classification (openvinotoolkit#2606)
- Fix saliency maps calculation issue for detection models (openvinotoolkit#2609)
- Fix h-label bug of missing parent labels in output (openvinotoolkit#2626)
- Re-introduce adaptive scheduling for training (openvinotoolkit#2541)
- Upgrade nncf version to 2.6.0 (openvinotoolkit#2459)
- Bump datumaro version to 1.5.0 (openvinotoolkit#2470, openvinotoolkit#2502)
- Set tox version constraint (openvinotoolkit#2472)
- Add model category attributes to model template (openvinotoolkit#2439)
- Bug fix for albumentations (openvinotoolkit#2467)
- Add workaround for the incorrect meta info M-RCNN (used for XAI) (openvinotoolkit#2437)
- Fix label list order for h-label classification (openvinotoolkit#2440)
- Modified fq numbers for lite HRNET e2e tests (openvinotoolkit#2445)
- Update the README file in exportable code (openvinotoolkit#2411)
- Fix broken links in documentation (openvinotoolkit#2405)
- Support encrypted dataset training (openvinotoolkit#2209)
- Add custom max iou assigner to prevent CPU OOM when large annotations are used (openvinotoolkit#2228)
- Auto train type detection for Semi-SL, Self-SL and Incremental: "--train-type" now is optional (openvinotoolkit#2195)
- Add per-class XAI saliency maps for Mask R-CNN model (openvinotoolkit#2227)
- Add new object detector Deformable DETR (openvinotoolkit#2249)
- Add new object detector DINO (openvinotoolkit#2266)
- Add new visual prompting task (openvinotoolkit#2203, openvinotoolkit#2274, openvinotoolkit#2311, openvinotoolkit#2354, openvinotoolkit#2318)
- Add new object detector ResNeXt101-ATSS (openvinotoolkit#2309)
- Introduce channel_last parameter to improve the performance (openvinotoolkit#2205)
- Decrease time for making a workspace (openvinotoolkit#2223)
- Set persistent_workers and pin_memory as True in detection task (openvinotoolkit#2224)
- New algorithm for Semi-SL semantic segmentation based on metric learning via class prototypes (openvinotoolkit#2156)
- Self-SL for classification now can recieve just folder with any images to start contrastive pretraining (openvinotoolkit#2219)
- Update OpenVINO version to 2023.0, and NNCF verion to 2.5 (openvinotoolkit#2090)
- Improve XAI saliency map generation for tiling detection and tiling instance segmentation (openvinotoolkit#2240)
- Remove CenterCrop from Classification test pipeline and editing missing docs link(openvinotoolkit#2375)
- Switch to PTQ for sseg (openvinotoolkit#2374)
- Fix the bug that auto adapt batch size is unavailable with IterBasedRunner (openvinotoolkit#2182)
- Fix the bug that learning rate isn't scaled when multi-GPU trianing is enabled(openvinotoolkit#2254)
- Fix the bug that label order is misaligned when model is deployed from Geti (openvinotoolkit#2369)
- Fix NNCF training on CPU (openvinotoolkit#2373)
- Fix H-label classification (openvinotoolkit#2377)
- Fix invalid import structures in otx.api (openvinotoolkit#2383)
- Add for async inference calculating saliency maps from predictions (Mask RCNN IR) (openvinotoolkit#2395)
- OpenVINO(==2023.0) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1
- n/a
- Fix a bug that auto adapt batch size doesn't work with cls incr case (openvinotoolkit#2199)
- Fix a bug that persistent worker is True even if num_workers is zero (openvinotoolkit#2208)
- OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)
- Support direct annotation input for COCO format (openvinotoolkit#1921)
- Action task supports multi GPU training. (openvinotoolkit#2057)
- Support storage cache in Apache Arrow using Datumaro for action tasks (openvinotoolkit#2087)
- Add a simplified greedy labels postprocessing for hierarchical classification (openvinotoolkit#2064).
- Support auto adapting batch size (openvinotoolkit#2119)
- Support auto adapting num_workers (openvinotoolkit#2165)
- Support noisy label detection for detection tasks (openvinotoolkit#2109, openvinotoolkit#2115, openvinotoolkit#2123, openvinotoolkit#2183)
- Make semantic segmentation OpenVINO models compatible with ModelAPI (openvinotoolkit#2029).
- Support label hierarchy through LabelTree in LabelSchema for classification task (openvinotoolkit#2149, openvinotoolkit#2152).
- Enhance exportable code file structure, video inference and default value for demo (openvinotoolkit#2051).
- Speedup OpenVINO inference in image classificaiton, semantic segmentation, object detection and instance segmentation tasks (openvinotoolkit#2105).
- Refactoring of ONNX export functionality (openvinotoolkit#2155).
- SSD detector Optimization(openvinotoolkit#2197)
- Fix async mode inference for demo in exportable code (openvinotoolkit#2154)
- Fix a bug that auto adapt batch size doesn't work with cls incr case (openvinotoolkit#2199)
- OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)
- Return raw anomaly map instead of colormap as metadata to prevent applying colormap conversion twice (openvinotoolkit#2217)
- Hotfix: use 0 confidence threshold when computing best threshold based on F1
- Improve warning message for tiling configurable parameter
- OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)
- Upgrade mmdeploy==0.14.0 from official PyPI (openvinotoolkit#2047)
- Integrate new ignored loss in semantic segmentation (openvinotoolkit#2065, openvinotoolkit#2111)
- Optimize YOLOX data pipeline (openvinotoolkit#2075)
- Tiling Spatial Concatenation for OpenVINO IR (openvinotoolkit#2052)
- Optimize counting train & inference speed and memory consumption (openvinotoolkit#2172)
- Bug fix: value of validation variable is changed after auto decrease batch size (openvinotoolkit#2053)
- Fix tiling 0 stride issue in parameter adapter (openvinotoolkit#2078)
- Fix Tiling NNCF (openvinotoolkit#2081)
- Do not skip full img tile classifier + Fix Sequencial Export Issue (openvinotoolkit#2174)
- Add generating feature cli_report.log in output for otx training (openvinotoolkit#1959)
- Support multiple python versions up to 3.10 (openvinotoolkit#1978)
- Support export of onnx models (openvinotoolkit#1976)
- Add option to save images after inference in OTX CLI demo together with demo in exportable code (openvinotoolkit#2005)
- Support storage cache in Apache Arrow using Datumaro for cls, det, seg tasks (openvinotoolkit#2009)
- Add noisy label detection for multi-class classification task (openvinotoolkit#1985, openvinotoolkit#2034)
- Add DeiT template for classification tasks as experimental template (<openvinotoolkit#2093)
- Clean up and refactor the output of the OTX CLI (openvinotoolkit#1946)
- Enhance DetCon logic and SupCon for semantic segmentation(openvinotoolkit#1958)
- Detection task refactoring (openvinotoolkit#1955)
- Classification task refactoring (openvinotoolkit#1972)
- Extend OTX explain CLI (openvinotoolkit#1941)
- Segmentation task refactoring (openvinotoolkit#1977)
- Action task refactoring (openvinotoolkit#1993)
- Optimize data preprocessing time and enhance overall performance in semantic segmentation (openvinotoolkit#2020)
- Support automatic batch size decrease when there is no enough GPU memory (openvinotoolkit#2022)
- Refine HPO usability (openvinotoolkit#2175)
- Fix backward compatibility with OpenVINO SSD-like detection models from OTE 0.5 (openvinotoolkit#1970)
- OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)
- Fix exception -> warning for anomaly dump_feature option
- Remove
dataset.with_empty_annotations()
to keep original input structure (openvinotoolkit#1964) - Fix OV batch inference (saliency map generation) (openvinotoolkit#1965)
- Replace EfficentNetB0 model download logic by pytorchcv to resolve zip issue (openvinotoolkit#1967)
- Add missing OpenVINO dependency in exportable code requirement
- Add FP16 IR export support (openvinotoolkit#1683)
- Add in-memory caching in dataloader (openvinotoolkit#1694)
- Add MoViNet template for action classification (openvinotoolkit#1742)
- Add Semi-SL multilabel classification algorithm (openvinotoolkit#1805)
- Integrate multi-gpu training for semi-supervised learning and self-supervised learning (openvinotoolkit#1534)
- Add train-type parameter to otx train (openvinotoolkit#1874)
- Add embedding of inference configuration to IR for classification (openvinotoolkit#1842)
- Enable VOC dataset in OTX (openvinotoolkit#1862)
- Add mmcls.VisionTransformer backbone support (openvinotoolkit#1908)
- Parametrize saliency maps dumping in export (openvinotoolkit#1888)
- Bring mmdeploy to action recognition model export & Test optimization of action tasks (openvinotoolkit#1848)
- Update backbone lists (openvinotoolkit#1835)
- Add explanation for XAI & minor doc fixes (openvinotoolkit#1923)
- Refactor phase#1: MPA modules
- Handle unpickable update_progress_callback (openvinotoolkit#1892)
- Dataset Adapter: Avoid duplicated annotation and permit empty image (openvinotoolkit#1873)
- Arrange scale between bbox preds and bbox targets in ATSS (openvinotoolkit#1880)
- Fix label mismatch of evaluation and validation with large dataset in semantic segmentation (openvinotoolkit#1851)
- Fix packaging errors including cython module build / import issues (openvinotoolkit#1936)
- OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)
- Refine documents by proof review
- Separate installation for each tasks
- Improve POT efficiency by setting stat_requests_number parameter to 1
- Introduce new tile classifier to enhance tiling inference performance in MaskRCNN.
- Fix missing classes in cls checkpoint
- Fix action task sample codes
- Fix label_scheme mismatch in classification
- Fix training error when batch size is 1
- Fix hang issue when tracing a stack in certain scenario
- Fix pickling error by Removing mmcv cfg dump in ckpt
NOTES
OpenVINO™ Training Extensions which version 1.0.0 has been updated to include functional and security updates. Users should update to the latest version.
- Adaptation of Datumaro component as a dataset interface
- Integrate hyper-parameter optimizations
- Support action recognition task
- Self-supervised learning mode for representational training
- Semi-supervised learning mode for better model quality
- Installation via PyPI package
- Enhance
find
command to find configurations of supported tasks / algorithms / models / backbones - Introduce
build
command to customize task or model configurations in isolated workspace - Auto-config feature to automatically select the right algorithm and default model for the
train
&build
command by detecting the task type of given input dataset - Improve documentation
- Improve training performance by introducing enhanced loss for the few-shot transfer
- Fixing configuration loading issue from the meta data of the model in OpenVINO task for the backward compatibility
- Fixing some minor issues
NOTES
OpenVINO Training Extension which version is equal or older then v0.5.0 does not include the latest functional and security updates. OTE Version 1.0.0 is targeted to be released in February 2023 and will include additional functional and security updates. Customers should update to the latest version as it becomes available.
- Add tiling in rotated detection (openvinotoolkit#1420)
- Add Cythonize AugMixAugment (openvinotoolkit#1478)
- Integrate ov-telemetry (openvinotoolkit#1568)
- Update OpenVINO to 2022.3 release & nncf to the pre-2.4 version (openvinotoolkit#1393)
- Fixing h-label head output bug in OV inference (openvinotoolkit#1458)
- Fixing deprecated np.bool issue from numpy==1.24.0 (openvinotoolkit#1455)
- Fixing tiling OpenVINO backward compatibility (openvinotoolkit#1516)
- Fixing indexing in hierarchical classification inference (openvinotoolkit#1551)
- Copying feature vector to resolve duplication issue (openvinotoolkit#1511)
- Fixing handling ignored samples in hierarchical head (openvinotoolkit#1599)
- Some minor issues
- Model Preparation Algorithm (MPA)
- Better saliency map support
- Replace current saliency map generation with Recipro-CAM for cls (openvinotoolkit#1363)
- Class-wise saliency map generation for the detection task (openvinotoolkit#1402)
- OTE Saliency Map Label (openvinotoolkit#1447)
- Improve object counting algorithm for high-res images via image tiling
- Add Tiling Module (openvinotoolkit#1200)
- Fliter object less than 1 pixel (openvinotoolkit#1305)
- Tiling deployment (openvinotoolkit#1387)
- Enable tiling oriented detection for v0.4.0/geti1.1.0 (openvinotoolkit#1427)
- Better saliency map support
- Hot-fix for Detection fix two stage error (openvinotoolkit#1433)
- Fixing ZeroDivisionError in iteration counter for detection-classification project trainings (openvinotoolkit#1449)
- Some minor issues
-
Neural Network Compression Framework (NNCF)
- Fix CUDA OOM for NNCF optimization model MaskRCNN-EfficientNetB2B (openvinotoolkit#1319)
-
Model Preparation Algorithm (MPA)
- Fix 'Shape out of bounds' error when accepting AI predictions for detection oriented (openvinotoolkit#1326)
- Fix weird confidence behaviour issue on predictions for hierarchical classification (openvinotoolkit#1332)
- Fix training failure issue for hierarchical classification (openvinotoolkit#1329)
- Fix training failure issues for segmentation and instance segmentation during inference process (openvinotoolkit#1338)
- Some minor issues
- Update vulnerable Python dependencies in OTE (openvinotoolkit#1303)
- Model Preparation Algorithm (MPA)
- Add new tasks / model templates for Class-Incremental Learning
- Instance Segmentation (openvinotoolkit#1142)
- Classification
- Multilabel (openvinotoolkit#1132)
- Hierarchical-label (openvinotoolkit#1159)
- SSD and YOLOX model template for Detection (openvinotoolkit#1156)
- Saliency map support
- Classification (openvinotoolkit#1166)
- Detection (openvinotoolkit#1155)
- Segmentation (openvinotoolkit#1158)
- NNCF (openvinotoolkit#1157) support
- HPO (openvinotoolkit#1168) support
- Balanced Sampler support for Classification (openvinotoolkit#1139)
- Add Adaptive Training for Detection / Instance Segmentation (openvinotoolkit#1190)
- Add new tasks / model templates for Class-Incremental Learning
- Anomaly
- Add real-life training tests (openvinotoolkit#898)
- Add additional check for early stopping parameter (openvinotoolkit#1110)
- Add DRAEM task implementation (openvinotoolkit#1203)
-
Model Preparation Algorithm (MPA)
- Replace Class-Incremental Learning models as OTE default models (openvinotoolkit#1150)
- Replace OTE ignored label support with external ignored label
- Classification (openvinotoolkit#1132)
- Detection (openvinotoolkit#1128)
- Segmentation (openvinotoolkit#1134)
- Enable mixed precision for Classification / Detection / Segmentation (openvinotoolkit#1198)
- Enhance training schedule for Classification (openvinotoolkit#1212)
- Change Model optimization hyper-parameters for Classification / Detection (openvinotoolkit#1170)
- Disable Obsolete test cases for OTE CI (openvinotoolkit#1220)
-
Anomaly
- Extend conftest configuration for anomaly backend (openvinotoolkit#1097)
- Expose more params to the UI (openvinotoolkit#1085)
- Change directory structure for anomaly templates (openvinotoolkit#1105)
- Use is_anomalous attribute instead of string matching (openvinotoolkit#1120)
- Set nncf version (openvinotoolkit#1124)
- Move to learning parameters (openvinotoolkit#1152)
- Change OpenVINO MO Command (openvinotoolkit#1221)
-
Model Preparation Algorithm (MPA)
- Fix inference issues for Detection (openvinotoolkit#1167)
- Fix model compatibility issue between SC1.1 and 1.2 in Segmentation (openvinotoolkit#1264)
- Some minor issues
-
Anomaly
- Fix non deterministic + sample.py (openvinotoolkit#1118)
- Fix exportable code for anomaly tasks (openvinotoolkit#1113)
- Fix local anomaly segmentation performance bug (openvinotoolkit#1219)
- Fix progress bar (openvinotoolkit#1223)
- Fix inference when model backbone changes (openvinotoolkit#1242)
- Model Preparation Algorithm (MPA), a newly introduced OTE Algorithm backend for advanced transfer learning
- Class-Incremental Learning support for OTE models
- Image Classification
- Object Detection
- Semantic Segmentation
- Class-Incremental Learning support for OTE models
- Object counting & Rotated object detection are added to Object Detection backend
- Increased support for NNCF / FP16 / HPO
- Ignored label support
- Stop training on NaN losses
- Major refactoring
- Tasks & model templates had been moved to OTE repo from each OTE Algorithm backend
- Some minor issues
- OTE SDK, defines an interface which can be used by OTE CLI to access OTE Algorithms.
- OTE CLI, contains set of commands needed to operate with deep learning models using OTE SDK Task interfaces.
- OTE Algorithms, contains sub-projects implementing OTE SDK Task interfaces for different deep learning models.
- Anomaly Classification
- Image Classification
- Object Detection
- Semantic Segmentation