[OTX] Apply changes in develop to feature/otx branch (#1436)

* Add tiling module (#1200)

* Update submodule branch (#1222)

* Enhance training schedule for multi-label classification (#1212)

* [CVS-88098] Remove initialize from export functions (#1226)

* Train graph added (#1211)

* Add @attrs decorator for base configs (#1229)

* Pretrained weight download error in MobilenetV3-large-1 of deep-object-reid in SC (#1233)

* [Anomaly Task] Revert hpo template (#1230)

* 🐞 [Anomaly Task] Fix progress bar (#1223)

* [CVS-90555] Fix NaN value in classification (#1244)

* update hpo_config.yaml (#1240)

* [CVS-90400, CVS-91015] NNCF pruning supported tweaks (#1248)

* [Anomaly Task] 🐞 Fix inference when model backbone changes (#1242)

* [CVS-91472] Add pruning_supported value (#1263)

* Pruning supported tweaks (#1256)

* [CVS-90400, CVS-91015] NNCF pruning supported tweaks (#1248)

* Revert "[CVS-90400, CVS-91015] NNCF pruning supported tweaks (#1248)" (#1269)

* [OTE-TEST] Disable obsolete test cases (#1220)

* [OTE-TEST] hot-fix for MPA performance tests (#1273)

* [Anomaly Task] ✨ Upgrade anomalib (#1243)

* Expose early stopping hyper-parameters for all tasks (#1241)

* Resolve pre-commit issues (#1272)

* Remove LazyEarlyStopHook in model_multilabel.py (#1281)

* Removed xfail (#1239)

* Implement IB loss for incremental learning in multi-class classification (#1289)

* Edit num_workers and change MPA repo as a latest (#1314)

* fix annotation bug (#1320)

* Valid POT configs for small HRNet models (#1313)

* Disable NNCF optimization for FP16 models (#1312)

* fliter object less than 1 pixel  (#1305)

* Fix some tests (#1322)

* [Develop] Move drop_last into MPA (#1357)

* Apply changes from releases/v0.3.1-geti1.0.0 (#1337)

* anomaly save_model bugfix (#1300)

* upgrade networkx module version (#1303)

* Forward CVS-94422 size bug fix PR to release branch (#1326)

* Valid POT configs for small HRNet models (#1317)

* [Release branch] Disable NNCF optimization for FP16 models  (#1319)

* [RELEASE] CVS-95549 - Hierarchical classification training failed without obvious reason (#1329)

* Fix h-label: per-group softmax (#1332)

* Fix dataset length bug in mpa task (#1338)

* Fix drop_last key issue for det/set (#1340)

* Hot-fix for OV inference for iseg output (#1345)

* Fix nncf model export bug (#1346)

* Fixed merge error (#1359)

* Update evaluation iou_thr of ins-seg (#1354)

* fix pre-commit test (#1366)

* Fix dataset item tests (#1360)

* Fix OV Inference issues (tiling tests & detection tests) (#1361)

* fix black & add xfail test cases (#1367)

* Update check_nncf_graph. (#1330)

* [Develop] Hot-fix OV inference issue in rotated detection (#1375)

* [Develop] updated documents (#1383)

* [CVS-94911] Fix difference between train and validation normalization pipeline (#1310)

* Update configs for padim model (#1378)

* updated QUICK_START_GUIDE.md (#1397)

* Change ote threshold of openvino test for cls (#1401)

* Normalize top-1 metrics to [0, 1] (#1394)

* Tiling deployment (#1387)

* Replace current saliency map generation with Recipro-CAM for cls (#1363)

* Class-wise saliency map generation for the detection task (#1402)

* Change submodule to develop (#1410)

* Send full dataset to POT optimization function (#1379) & Convert NaN to num to make visible in geti UI (#1413)

* Add active score evaluation to the classification task

* [release/0.4.0][OTX] Enabling GPU execution for exported code (#1416)

* [OTE][Release][XAI] Detection fix two stage bbox_head error (#1414)

* Update SDK commit for exportable code (#1423)

* HRNet-x and HRNe-18--mod2 configs update (#1419)

* [Release] Enable tiling oriented detection for v0.4.0/geti1.1.0 (#1427)

* [OTE][Releases v0.4.0][XAI] Hot-fix for Detection fix two stage error (#1433)

* Temporary MPA branch while dev->otx merge process

* Update doc & install for dev->otx changes

* Update ote_sdk -> otx.api

* Update ote_cli -> otx.cli

* Update external/mmsegmentation -> otx/algorithms/segmentation

* Align saliency map media instantiation over tasks (#1447)

* Update external/d-o-r -> otx/algorithms/classification

* Update external/mmdetection -> otx/algorithms/detection

* Update external/mpa -> otx/algorithms/*

* Fix CLI test run for better error message

* Numpy constraint for deprecated np.bool error

* Capture stderr only

* Align numpy requirement

* [OTX/Anomaly] Add changes from external to otx (#1452)

* Add changes from external to otx

* Address PR comments

* Update config files + remove backbone from base

* Fix pre-merge checks

* Fix pre-commit issues

* Update exportable code commit

* Fix indent error

* Fix flake8 issue

* Resolve softmax issue w/ FIXME for future work

* Add tiling tests

* Revert MPA branch to otx

Signed-off-by: Songki Choi <songki.choi@intel.com>
Co-authored-by: Eugene Liu <eugene.liu@intel.com>
Co-authored-by: Ashwin Vaidya <ashwin.vaidya@intel.com>
Co-authored-by: Jaeguk Hyun <jaeguk.hyun@intel.com>
Co-authored-by: Nikita Savelyev <nikita.savelyev@intel.com>
Co-authored-by: Jihwan Eom <jihwan.eom@intel.com>
Co-authored-by: Harim Kang <harim.kang@intel.com>
Co-authored-by: Soobee Lee <soobee.lee@intel.com>
Co-authored-by: Lee, Soobee <soobeele@intel.com>
Co-authored-by: Emily Chun <emily.chun@intel.com>
Co-authored-by: ljcornel <ludo.cornelissen@intel.com>
Co-authored-by: Eunwoo Shin <eunwoo.shin@intel.com>
Co-authored-by: dlyakhov <daniil.lyakhov@intel.com>
Co-authored-by: kprokofi <kirill.prokofiev@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Yunchu Lee <yunchu.lee@intel.com>
Co-authored-by: Ashwin Vaidya <ashwinitinvaidya@gmail.com>
Co-authored-by: Alexander Dokuchaev <alexander.dokuchaev@intel.com>
Co-authored-by: Vladislav Sovrasov <vladislav.sovrasov@intel.com>
Co-authored-by: Evgeny Tsykunov <e.tsykunov@gmail.com>
Co-authored-by: Galina Zalesskaya <galina.zalesskaya@intel.com>
Co-authored-by: dongkwan-kim <dongkwan.kim@intel.com>

.gitignore

@@ -4,11 +4,12 @@ __pycache__
 .vscode/
 *.iml
 
-venv
+*venv*/
 env
 otx-workspace*
 .env
 .tox
+results/
 
 data/*
 .coverage

.pre-commit-config.yaml

@@ -11,18 +11,6 @@ repos:
         files: '^(otx|tests)/.*\.py'
         exclude: "tests/ote_cli"
 
-      # NOTE: ote-sdk and ote-cli will be deprecated. The following config
-      # is to be removed.
-      - id: isort
-        alias: isort_ote_sdk
-        name: "isort - legacy (ote_sdk)"
-        files: '^ote_sdk/.*\.py'
-      - id: isort
-        alias: isort_rest
-        name: "isort (ote_cli|external)"
-        files: '^(ote_cli|external/anomaly|external/model-preparation-algorithm)/.*\.py'
-        exclude: "tests/"
-
   - repo: https://github.com/psf/black
     rev: 22.6.0
     hooks:
@@ -31,32 +19,21 @@ repos:
         files: '^(otx|tests)/.*\.py'
         exclude: "tests/ote_cli"
 
-      # NOTE: ote-sdk and ote-cli will be deprecated. The following config
-      # is to be removed.
-      - id: black
-        name: "black - legacy (ote_sdk|ote_cli)"
-        args: [--line-length, "88"]
-        files: '^(ote_sdk|ote_cli)/.*\.py'
-      - id: black
-        name: "black - legacy (rest)"
-        args: [--line-length, "120"]
-        files: '^(external/anomaly|external/model-preparation-algorithm)/.*\.py'
-
   - repo: https://github.com/PyCQA/flake8
     rev: "5.0.3"
     hooks:
       - id: flake8
         name: "flake8"
-        files: '^(ote_sdk|ote_cli|external/anomaly|external/model-preparation-algorithm)/.*\.py'
+        files: '^(otx|tests)/.*\.py'
         args: ["--config", ".flake8", "--max-complexity", "20"]
-        exclude: ".*/protobuf"
+        exclude: "^(.*/protobuf|tests/ote_cli)"
 
   # yaml formatting
   - repo: https://github.com/pre-commit/mirrors-prettier
     rev: v2.7.1
     hooks:
       - id: prettier
-        exclude: "external/deep-object-reid"
+        exclude: "external"
 
   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: "v0.971"
@@ -74,35 +51,6 @@ repos:
           - types-python-dateutil
         exclude: "^otx/algorithms/anomaly/tests"
 
-      # NOTE: ote-sdk and ote-cli will be deprecated. The following config
-      # is to be removed.
-      - id: mypy
-        alias: mypy_ote_sdk
-        name: "mypy - legacy (ote_sdk)"
-        files: '^ote_sdk/.*\.py'
-        additional_dependencies:
-          [
-            numpy==1.19.5,
-            types-PyYAML,
-            attrs==21.2.*,
-            types-requests,
-            types-Deprecated,
-            types-docutils,
-            types_futures,
-            types-python-dateutil,
-          ]
-      - id: mypy
-        alias: mypy_ote_cli
-        name: "mypy - legacy (ote_cli)"
-        files: '^ote_cli/.*\.py'
-        additional_dependencies: [types-PyYAML]
-      - id: mypy
-        alias: mypy_anomaly
-        name: "mypy (anomaly)"
-        files: '^external/anomaly/.*\.py'
-        additional_dependencies: [attrs==21.2.*, types-PyYAML]
-        exclude: "^external/anomaly/tests"
-
   - repo: https://github.com/PyCQA/pylint
     rev: "v2.14.5"
     hooks:
@@ -115,17 +63,6 @@ repos:
         types: [python]
         args: ["--score=no"]
 
-      # NOTE: ote-sdk and ote-cli will be deprecated. The following config
-      # is to be removed.
-      - id: pylint
-        name: "pylint - legacy"
-        files: '^(ote_sdk|ote_cli|external/anomaly)/.*\.py'
-        entry: pylint
-        language: system
-        types: [python]
-        args: ["--score=no"]
-        exclude: "tests"
-
   - repo: https://github.com/PyCQA/pydocstyle
     rev: 6.1.1
     hooks:
@@ -138,12 +75,13 @@ repos:
         files: '^otx/.*\.py'
         exclude: "otx/algorithms/anomaly/tests|external/anomaly/tests|otx/cli/utils/tests.py"
 
-  - repo: https://github.com/jumanjihouse/pre-commit-hooks
-    rev: 2.1.6
-    hooks:
-      - id: markdownlint
-        # TODO: Check all files even tests after migration to otx is complete
-        exclude: "^(ote_sdk|tests|.github)"
+    # Will use rst
+    # - repo: https://github.com/jumanjihouse/pre-commit-hooks
+    #   rev: 2.1.6
+    #   hooks:
+    #     - id: markdownlint
+    #       # TODO: Check all files even tests after migration to otx is complete
+    #       exclude: "^(ote_sdk|tests|.github|external)"
 
   - repo: https://github.com/AleksaC/hadolint-py
     rev: v2.10.0

CHANGELOG.md

@@ -2,6 +2,44 @@
 
 All notable changes to this project will be documented in this file.
 
+## \[v0.4.0\]
+
+### Added
+
+- Model Preparation Algorithm (MPA)
+  - Better saliency map support
+    - Replace current saliency map generation with Recipro-CAM for cls (<https://github.com/openvinotoolkit/training_extensions/pull/1363>)
+    - Class-wise saliency map generation for the detection task (<https://github.com/openvinotoolkit/training_extensions/pull/1402>)
+  - Improve object counting algorithm for high-res images via image tiling
+    - Add Tiling Module (<https://github.com/openvinotoolkit/training_extensions/pull/1200>)
+    - Fliter object less than 1 pixel (<https://github.com/openvinotoolkit/training_extensions/pull/1305>)
+    - Tiling deployment (<https://github.com/openvinotoolkit/training_extensions/pull/1387>)
+    - Enable tiling oriented detection for v0.4.0/geti1.1.0 (<https://github.com/openvinotoolkit/training_extensions/pull/1427>)
+
+### Fixed
+
+- Hot-fix for Detection fix two stage error (<https://github.com/openvinotoolkit/training_extensions/pull/1433>)
+- Some minor issues
+
+## \[v0.3.1\]
+
+### Fixed
+
+- Neural Network Compression Framework (NNCF)
+
+  - Fix CUDA OOM for NNCF optimization model MaskRCNN-EfficientNetB2B (<https://github.com/openvinotoolkit/training_extensions/pull/1319>)
+
+- Model Preparation Algorithm (MPA)
+  - Fix 'Shape out of bounds' error when accepting AI predictions for detection oriented (<https://github.com/openvinotoolkit/training_extensions/pull/1326>)
+  - Fix weird confidence behaviour issue on predictions for hierarchical classification (<https://github.com/openvinotoolkit/training_extensions/pull/1332>)
+  - Fix training failure issue for hierarchical classification (<https://github.com/openvinotoolkit/training_extensions/pull/1329>)
+  - Fix training failure issues for segmentation and instance segmentation during inference process (<https://github.com/openvinotoolkit/training_extensions/pull/1338>)
+  - Some minor issues
+
+### Security
+
+- Update vulnerable Python dependencies in OTE (<https://github.com/openvinotoolkit/training_extensions/pull/1303>)
+
 ## \[v0.3.0\]
 
 ### Added