CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

[v1.5.0 - unreleased]

New features

• Enable configurable confidence threshold for otx eval and export(openvinotoolkit#2388)
• Add YOLOX variants as new object detector models (openvinotoolkit#2402)
• Enable FeatureVectorHook to support action tasks(openvinotoolkit#2408)
• Add ONNX metadata to detection, instance segmantation, and segmentation models (openvinotoolkit#2418)
• Add a new feature to configure input size(openvinotoolkit#2420)
• Introduce the OTXSampler and AdaptiveRepeatDataHook to achieve faster training at the small data regime (openvinotoolkit#2428)
• Add a new object detector Lite-DINO(openvinotoolkit#2457)
• Add Semi-SL Mean Teacher algorithm for Instance Segmentation task(openvinotoolkit#2444)
• Official supports for YOLOX-X, YOLOX-L, YOLOX-S, ResNeXt101-ATSS (openvinotoolkit#2485)
• Add new argument to track resource usage in train command (openvinotoolkit#2500)
• Add Self-SL for semantic segmentation of SegNext families (openvinotoolkit#2215)
• Adapt input size automatically based on dataset statistics (openvinotoolkit#2499)

Enhancements

• Refine input data in-memory caching (openvinotoolkit#2416)
• Adapt timeout value of initialization for distributed training (openvinotoolkit#2422)
• Optimize data loading by merging load & resize operations w/ caching support for cls/det/iseg/sseg (openvinotoolkit#2438, openvinotoolkit#2453, openvinotoolkit#2460)
• Support torch==2.0.1 (openvinotoolkit#2465)
• Set "Auto" as default input size mode (openvinotoolkit#2515)

Bug fixes

• Fix F1 auto-threshold to choose best largest confidence (openvinotoolkit#2371)

Known issues

• OpenVINO(==2023.0) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch>=1.13.1
• NNCF QAT optimization is disabled for MaskRCNN models due to CUDA runtime error in ROIAlign kernel on torch==2.0.1

[v1.4.2]

Enhancements

• Bump datumaro version to 1.5.0rc0 (openvinotoolkit#2470)
• Set tox version constraint (openvinotoolkit#2472)
• Add model category attributes to model template (openvinotoolkit#2439)

Bug fixes

• Bug fix for albumentations (openvinotoolkit#2467)
• Add workaround for the incorrect meta info M-RCNN (used for XAI) (openvinotoolkit#2437)
• Fix label list order for h-label classification (openvinotoolkit#2440)
• Modified fq numbers for lite HRNET e2e tests (openvinotoolkit#2445)

[v1.4.1]

Enhancements

• Update the README file in exportable code (openvinotoolkit#2411)

Bug fixes

• Fix broken links in documentation (openvinotoolkit#2405)

[v1.4.0]

New features

• Support encrypted dataset training (openvinotoolkit#2209)
• Add custom max iou assigner to prevent CPU OOM when large annotations are used (openvinotoolkit#2228)
• Auto train type detection for Semi-SL, Self-SL and Incremental: "--train-type" now is optional (openvinotoolkit#2195)
• Add per-class XAI saliency maps for Mask R-CNN model (openvinotoolkit#2227)
• Add new object detector Deformable DETR (openvinotoolkit#2249)
• Add new object detector DINO (openvinotoolkit#2266)
• Add new visual prompting task (openvinotoolkit#2203, openvinotoolkit#2274, openvinotoolkit#2311, openvinotoolkit#2354, openvinotoolkit#2318)
• Add new object detector ResNeXt101-ATSS (openvinotoolkit#2309)

Enhancements

• Introduce channel_last parameter to improve the performance (openvinotoolkit#2205)
• Decrease time for making a workspace (openvinotoolkit#2223)
• Set persistent_workers and pin_memory as True in detection task (openvinotoolkit#2224)
• New algorithm for Semi-SL semantic segmentation based on metric learning via class prototypes (openvinotoolkit#2156)
• Self-SL for classification now can recieve just folder with any images to start contrastive pretraining (openvinotoolkit#2219)
• Update OpenVINO version to 2023.0, and NNCF verion to 2.5 (openvinotoolkit#2090)
• Improve XAI saliency map generation for tiling detection and tiling instance segmentation (openvinotoolkit#2240)
• Remove CenterCrop from Classification test pipeline and editing missing docs link(openvinotoolkit#2375)
• Switch to PTQ for sseg (openvinotoolkit#2374)

Bug fixes

• Fix the bug that auto adapt batch size is unavailable with IterBasedRunner (openvinotoolkit#2182)
• Fix the bug that learning rate isn't scaled when multi-GPU trianing is enabled(openvinotoolkit#2254)
• Fix the bug that label order is misaligned when model is deployed from Geti (openvinotoolkit#2369)
• Fix NNCF training on CPU (openvinotoolkit#2373)
• Fix H-label classification (openvinotoolkit#2377)
• Fix invalid import structures in otx.api (openvinotoolkit#2383)
• Add for async inference calculating saliency maps from predictions (Mask RCNN IR) (openvinotoolkit#2395)

Known issues

• OpenVINO(==2023.0) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1

[v1.3.1]

Enhancements

• n/a

Bug fixes

• Fix a bug that auto adapt batch size doesn't work with cls incr case (openvinotoolkit#2199)
• Fix a bug that persistent worker is True even if num_workers is zero (openvinotoolkit#2208)

Known issues

• OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)

[v1.3.0]

New features

• Support direct annotation input for COCO format (openvinotoolkit#1921)
• Action task supports multi GPU training. (openvinotoolkit#2057)
• Support storage cache in Apache Arrow using Datumaro for action tasks (openvinotoolkit#2087)
• Add a simplified greedy labels postprocessing for hierarchical classification (openvinotoolkit#2064).
• Support auto adapting batch size (openvinotoolkit#2119)
• Support auto adapting num_workers (openvinotoolkit#2165)
• Support noisy label detection for detection tasks (openvinotoolkit#2109, openvinotoolkit#2115, openvinotoolkit#2123, openvinotoolkit#2183)

Enhancements

• Make semantic segmentation OpenVINO models compatible with ModelAPI (openvinotoolkit#2029).
• Support label hierarchy through LabelTree in LabelSchema for classification task (openvinotoolkit#2149, openvinotoolkit#2152).
• Enhance exportable code file structure, video inference and default value for demo (openvinotoolkit#2051).
• Speedup OpenVINO inference in image classificaiton, semantic segmentation, object detection and instance segmentation tasks (openvinotoolkit#2105).
• Refactoring of ONNX export functionality (openvinotoolkit#2155).
• SSD detector Optimization(openvinotoolkit#2197)

Bug fixes

• Fix async mode inference for demo in exportable code (openvinotoolkit#2154)
• Fix a bug that auto adapt batch size doesn't work with cls incr case (openvinotoolkit#2199)

Known issues

• OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)

[v1.2.3]

Bug fixes

• Return raw anomaly map instead of colormap as metadata to prevent applying colormap conversion twice (openvinotoolkit#2217)
• Hotfix: use 0 confidence threshold when computing best threshold based on F1

[v1.2.2]

Enhancements

• Improve warning message for tiling configurable parameter

Known issues

• OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)

[v1.2.1]

Enhancements

• Upgrade mmdeploy==0.14.0 from official PyPI (openvinotoolkit#2047)
• Integrate new ignored loss in semantic segmentation (openvinotoolkit#2065, openvinotoolkit#2111)
• Optimize YOLOX data pipeline (openvinotoolkit#2075)
• Tiling Spatial Concatenation for OpenVINO IR (openvinotoolkit#2052)
• Optimize counting train & inference speed and memory consumption (openvinotoolkit#2172)

Bug fixes

• Bug fix: value of validation variable is changed after auto decrease batch size (openvinotoolkit#2053)
• Fix tiling 0 stride issue in parameter adapter (openvinotoolkit#2078)
• Fix Tiling NNCF (openvinotoolkit#2081)
• Do not skip full img tile classifier + Fix Sequencial Export Issue (openvinotoolkit#2174)

[v1.2.0]

New features

• Add generating feature cli_report.log in output for otx training (openvinotoolkit#1959)
• Support multiple python versions up to 3.10 (openvinotoolkit#1978)
• Support export of onnx models (openvinotoolkit#1976)
• Add option to save images after inference in OTX CLI demo together with demo in exportable code (openvinotoolkit#2005)
• Support storage cache in Apache Arrow using Datumaro for cls, det, seg tasks (openvinotoolkit#2009)
• Add noisy label detection for multi-class classification task (openvinotoolkit#1985, openvinotoolkit#2034)
• Add DeiT template for classification tasks as experimental template (<openvinotoolkit#2093)

Enhancements

• Clean up and refactor the output of the OTX CLI (openvinotoolkit#1946)
• Enhance DetCon logic and SupCon for semantic segmentation(openvinotoolkit#1958)
• Detection task refactoring (openvinotoolkit#1955)
• Classification task refactoring (openvinotoolkit#1972)
• Extend OTX explain CLI (openvinotoolkit#1941)
• Segmentation task refactoring (openvinotoolkit#1977)
• Action task refactoring (openvinotoolkit#1993)
• Optimize data preprocessing time and enhance overall performance in semantic segmentation (openvinotoolkit#2020)
• Support automatic batch size decrease when there is no enough GPU memory (openvinotoolkit#2022)
• Refine HPO usability (openvinotoolkit#2175)

Bug fixes

• Fix backward compatibility with OpenVINO SSD-like detection models from OTE 0.5 (openvinotoolkit#1970)

Known issues

• OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)

[v1.1.2]

Bug fixes

• Fix exception -> warning for anomaly dump_feature option
• Remove dataset.with_empty_annotations() to keep original input structure (openvinotoolkit#1964)
• Fix OV batch inference (saliency map generation) (openvinotoolkit#1965)
• Replace EfficentNetB0 model download logic by pytorchcv to resolve zip issue (openvinotoolkit#1967)

[v1.1.1]

Bug fixes

• Add missing OpenVINO dependency in exportable code requirement

[v1.1.0]

New features

• Add FP16 IR export support (openvinotoolkit#1683)
• Add in-memory caching in dataloader (openvinotoolkit#1694)
• Add MoViNet template for action classification (openvinotoolkit#1742)
• Add Semi-SL multilabel classification algorithm (openvinotoolkit#1805)
• Integrate multi-gpu training for semi-supervised learning and self-supervised learning (openvinotoolkit#1534)
• Add train-type parameter to otx train (openvinotoolkit#1874)
• Add embedding of inference configuration to IR for classification (openvinotoolkit#1842)
• Enable VOC dataset in OTX (openvinotoolkit#1862)
• Add mmcls.VisionTransformer backbone support (openvinotoolkit#1908)

Enhancements

• Parametrize saliency maps dumping in export (openvinotoolkit#1888)
• Bring mmdeploy to action recognition model export & Test optimization of action tasks (openvinotoolkit#1848)
• Update backbone lists (openvinotoolkit#1835)
• Add explanation for XAI & minor doc fixes (openvinotoolkit#1923)
• Refactor phase#1: MPA modules

Bug fixes

• Handle unpickable update_progress_callback (openvinotoolkit#1892)
• Dataset Adapter: Avoid duplicated annotation and permit empty image (openvinotoolkit#1873)
• Arrange scale between bbox preds and bbox targets in ATSS (openvinotoolkit#1880)
• Fix label mismatch of evaluation and validation with large dataset in semantic segmentation (openvinotoolkit#1851)
• Fix packaging errors including cython module build / import issues (openvinotoolkit#1936)

Known issues

• OpenVINO(==2022.3) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1 (working well up to torch==1.12.1) (openvinotoolkit#1906)

[v1.0.1]

Enhancements

• Refine documents by proof review
• Separate installation for each tasks
• Improve POT efficiency by setting stat_requests_number parameter to 1
• Introduce new tile classifier to enhance tiling inference performance in MaskRCNN.

Bug fixes

• Fix missing classes in cls checkpoint
• Fix action task sample codes
• Fix label_scheme mismatch in classification
• Fix training error when batch size is 1
• Fix hang issue when tracing a stack in certain scenario
• Fix pickling error by Removing mmcv cfg dump in ckpt

[v1.0.0]

NOTES

OpenVINO™ Training Extensions which version 1.0.0 has been updated to include functional and security updates. Users should update to the latest version.

New features

• Adaptation of Datumaro component as a dataset interface
• Integrate hyper-parameter optimizations
• Support action recognition task
• Self-supervised learning mode for representational training
• Semi-supervised learning mode for better model quality

Enhancements

• Installation via PyPI package
• Enhance find command to find configurations of supported tasks / algorithms / models / backbones
• Introduce build command to customize task or model configurations in isolated workspace
• Auto-config feature to automatically select the right algorithm and default model for the train & build command by detecting the task type of given input dataset
• Improve documentation
• Improve training performance by introducing enhanced loss for the few-shot transfer

Bug fixes

• Fixing configuration loading issue from the meta data of the model in OpenVINO task for the backward compatibility
• Fixing some minor issues

[v0.5.0]

NOTES

OpenVINO Training Extension which version is equal or older then v0.5.0 does not include the latest functional and security updates. OTE Version 1.0.0 is targeted to be released in February 2023 and will include additional functional and security updates. Customers should update to the latest version as it becomes available.

Added

• Add tiling in rotated detection (openvinotoolkit#1420)
• Add Cythonize AugMixAugment (openvinotoolkit#1478)
• Integrate ov-telemetry (openvinotoolkit#1568)

Changed

• Update OpenVINO to 2022.3 release & nncf to the pre-2.4 version (openvinotoolkit#1393)

Fixed

• Fixing h-label head output bug in OV inference (openvinotoolkit#1458)
• Fixing deprecated np.bool issue from numpy==1.24.0 (openvinotoolkit#1455)
• Fixing tiling OpenVINO backward compatibility (openvinotoolkit#1516)
• Fixing indexing in hierarchical classification inference (openvinotoolkit#1551)
• Copying feature vector to resolve duplication issue (openvinotoolkit#1511)
• Fixing handling ignored samples in hierarchical head (openvinotoolkit#1599)
• Some minor issues

[v0.4.0]

Added

• Model Preparation Algorithm (MPA)
  • Better saliency map support
    • Replace current saliency map generation with Recipro-CAM for cls (openvinotoolkit#1363)
    • Class-wise saliency map generation for the detection task (openvinotoolkit#1402)
    • OTE Saliency Map Label (openvinotoolkit#1447)
  • Improve object counting algorithm for high-res images via image tiling
    • Add Tiling Module (openvinotoolkit#1200)
    • Fliter object less than 1 pixel (openvinotoolkit#1305)
    • Tiling deployment (openvinotoolkit#1387)
    • Enable tiling oriented detection for v0.4.0/geti1.1.0 (openvinotoolkit#1427)

Fixed

• Hot-fix for Detection fix two stage error (openvinotoolkit#1433)
• Fixing ZeroDivisionError in iteration counter for detection-classification project trainings (openvinotoolkit#1449)
• Some minor issues

[v0.3.1]

Fixed

• Neural Network Compression Framework (NNCF)

  • Fix CUDA OOM for NNCF optimization model MaskRCNN-EfficientNetB2B (openvinotoolkit#1319)

• Model Preparation Algorithm (MPA)

  • Fix 'Shape out of bounds' error when accepting AI predictions for detection oriented (openvinotoolkit#1326)
  • Fix weird confidence behaviour issue on predictions for hierarchical classification (openvinotoolkit#1332)
  • Fix training failure issue for hierarchical classification (openvinotoolkit#1329)
  • Fix training failure issues for segmentation and instance segmentation during inference process (openvinotoolkit#1338)
  • Some minor issues

Security

• Update vulnerable Python dependencies in OTE (openvinotoolkit#1303)

[v0.3.0]

Added

• Model Preparation Algorithm (MPA)
  • Add new tasks / model templates for Class-Incremental Learning
    • Instance Segmentation (openvinotoolkit#1142)
    • Classification
      • Multilabel (openvinotoolkit#1132)
      • Hierarchical-label (openvinotoolkit#1159)
    • SSD and YOLOX model template for Detection (openvinotoolkit#1156)
  • Saliency map support
    • Classification (openvinotoolkit#1166)
    • Detection (openvinotoolkit#1155)
    • Segmentation (openvinotoolkit#1158)
  • NNCF (openvinotoolkit#1157) support
  • HPO (openvinotoolkit#1168) support
  • Balanced Sampler support for Classification (openvinotoolkit#1139)
  • Add Adaptive Training for Detection / Instance Segmentation (openvinotoolkit#1190)
• Anomaly
  • Add real-life training tests (openvinotoolkit#898)
  • Add additional check for early stopping parameter (openvinotoolkit#1110)
  • Add DRAEM task implementation (openvinotoolkit#1203)

Changed

• Model Preparation Algorithm (MPA)

  • Replace Class-Incremental Learning models as OTE default models (openvinotoolkit#1150)
  • Replace OTE ignored label support with external ignored label
    • Classification (openvinotoolkit#1132)
    • Detection (openvinotoolkit#1128)
    • Segmentation (openvinotoolkit#1134)
  • Enable mixed precision for Classification / Detection / Segmentation (openvinotoolkit#1198)
  • Enhance training schedule for Classification (openvinotoolkit#1212)
  • Change Model optimization hyper-parameters for Classification / Detection (openvinotoolkit#1170)
  • Disable Obsolete test cases for OTE CI (openvinotoolkit#1220)

• Anomaly

  • Extend conftest configuration for anomaly backend (openvinotoolkit#1097)
  • Expose more params to the UI (openvinotoolkit#1085)
  • Change directory structure for anomaly templates (openvinotoolkit#1105)
  • Use is_anomalous attribute instead of string matching (openvinotoolkit#1120)
  • Set nncf version (openvinotoolkit#1124)
  • Move to learning parameters (openvinotoolkit#1152)
  • Change OpenVINO MO Command (openvinotoolkit#1221)

Fixed

• Model Preparation Algorithm (MPA)

  • Fix inference issues for Detection (openvinotoolkit#1167)
  • Fix model compatibility issue between SC1.1 and 1.2 in Segmentation (openvinotoolkit#1264)
  • Some minor issues

• Anomaly

  • Fix non deterministic + sample.py (openvinotoolkit#1118)
  • Fix exportable code for anomaly tasks (openvinotoolkit#1113)
  • Fix local anomaly segmentation performance bug (openvinotoolkit#1219)
  • Fix progress bar (openvinotoolkit#1223)
  • Fix inference when model backbone changes (openvinotoolkit#1242)

[v0.2.0]

Added

• Model Preparation Algorithm (MPA), a newly introduced OTE Algorithm backend for advanced transfer learning
  • Class-Incremental Learning support for OTE models
    • Image Classification
    • Object Detection
    • Semantic Segmentation
• Object counting & Rotated object detection are added to Object Detection backend
• Increased support for NNCF / FP16 / HPO
• Ignored label support
• Stop training on NaN losses

Changed

• Major refactoring
  • Tasks & model templates had been moved to OTE repo from each OTE Algorithm backend

[v0.1.1]

Fixed

• Some minor issues

[v0.1.0]

Added

• OTE SDK, defines an interface which can be used by OTE CLI to access OTE Algorithms.
• OTE CLI, contains set of commands needed to operate with deep learning models using OTE SDK Task interfaces.
• OTE Algorithms, contains sub-projects implementing OTE SDK Task interfaces for different deep learning models.
  • Anomaly Classification
  • Image Classification
  • Object Detection
  • Semantic Segmentation