Mergeback 2.1.0rc0 to develop (#3694)

* [hotfix] Update for fix workflow issues (#3668) * Update README & CODEOWNERS (#3659) * Update Engine's docstring & CLI --help outputs (#3658) Update Engine's CLI docstring & HelpFormatter * Fix unit test for semantic segmentation to run it without mmseg (#3670) fix test to run wo mm * Enable ruff & ruff-format into otx/algo/classification/backbones (#3667) * Enable ruff in otx.algo.classification.backbones * Fix unit-test * Fix docstring * Disable integration test in pr-merge workflow (#3677) * Add TV MaskRCNN Tile Recipe (#3655) * add tv maskrcnn recipe * add unit test * add tile size * fix bug * Align integration test to exportable code interface update for release branch (#3676) * align integration test to exportable code update * add error message * align vp expected output name * Refactor exporter for anomaly task and fix a bug with exportable code (#3672) * refactor exporter * remove *to * reply comments * resolve problem with local temp file * Fix NNCF MaskRCNN-Eff accuracy drop (#3680) * fix nncf maskrcnn eff accuracy drop * update ignored scope * fix typo * Update pandas version constraint (#3679) * Include more models to export test into test_otx_e2e (#3678) * enable export test * re-enable failed models * exclude exportable code test from anomaly task * apply it to e2e test code * align with pre-commit * Fix optimize with Semi-SL data pipeline (#3684) Disable unlabeled dataset with optimize * Fix MaskRCNN SwinT NNCF Accuracy Drop (#3685) fix swin-t nncf kinda * Bump MAPI version (#3686) * Add rotated det OV recipe (#3687) * Add rotated det ov recipe * fix e2e test cli * Move assigning tasks to Models from Engine to Anomaly Model Classes (#3683) * Move Task assign into Model with Anomaly Task * Fix openvino model class * Refactoring detection modules (#3636) * Organize common functions that are located in each det model into OTXDetectionModel - __init__, _build_model, _create_model, _customize_inputs, _customize_outputs, get_classification_layers * Create common directory for modules used across tasks * Remove DictConfig to use dict instead * Update docstring * Refactoring - export related things - grouped importing backbones, necks, heads, losses - move assigners, coders, prior_generators, samplers which are located in heads to utils * Fix segmentation fault on VPM PTQ (#3689) Add forcing `num_workers` to 0 * Fix detection performance degradation (#3691) Update to apply filtering to only `test_step` and `predict_step` not to affect training * Bump datumaro to 1.7.0 (#3669) * bump datumaro to 1.7.0 * apply changes on datumaro 1.7.0 * Add maskrcnn_r50_tv_tile (#3692) * Update version string and fill missing changelog (#3693) --------- Co-authored-by: Harim Kang <harim.kang@intel.com> Co-authored-by: Prokofiev Kirill <kirill.prokofiev@intel.com> Co-authored-by: Eugene Liu <eugene.liu@intel.com> Co-authored-by: Eunwoo Shin <eunwoo.shin@intel.com> Co-authored-by: Vladislav Sovrasov <sovrasov.vlad@gmail.com> Co-authored-by: Kim, Sungchul <sungchul.kim@intel.com> Co-authored-by: Emily Chun <emily.chun@intel.com>
openvinotoolkit · Jun 29, 2024 · c51a787 · c51a787
1 parent ba875e1
commit c51a787
Show file tree

Hide file tree

Showing 131 changed files with 2,302 additions and 2,311 deletions.
diff --git a/.ci/docker/README.md b/.ci/docker/README.md
@@ -5,7 +5,7 @@
 
    ```bash
    training_extensions/.ci/docker$ ./build.sh --help
-      USAGE: .ci/docker/build.sh <tag> [Options]
+      USAGE: ./build.sh <tag> [Options]
       Positional args
          <tag>               Tag name to be tagged to newly built image
       Options
@@ -18,7 +18,7 @@
    Below example builds an image using actions-runner v2.317.0 based on NVIDIA cuda image and tag it as `2.317.0`.
 
    ```bash
-   training_extensions$ .ci/build.sh 2.317.0 -u https://github.com/actions/runner/releases/download/v2.305.0/actions-runner-linux-x64-2.305.0.tar.gz
+   training_extensions/.ci/docker$ ./build.sh 2.317.0 -u https://github.com/actions/runner/releases/download/v2.305.0/actions-runner-linux-x64-2.305.0.tar.gz
    ```
 
    > **_Note_**: While building an image, script will use your system's environment variables `http_proxy`, `https_proxy`, and `no_proxy`. If you need to use proxy to access external entity, please check those settings before using this script.
@@ -55,7 +55,7 @@
    Below example starts a runner named as `<container-prefix>-0` with GPU ID 0 (GPU ID will be attached to both container and runner name)
 
    ```bash
-   training_extensions$ .ci/start-runner.sh <container-prefix> <github-token> -g 0
+   training_extensions$ .ci/docker/start-runner.sh <container-prefix> <github-token> -g 0
    ```
 
    If there exist the container named as same, it will be stopped before starting a new container.
@@ -86,14 +86,14 @@
    Below example stops a runner named as `otx-ci-container`
 
    ```bash
-   training_extensions$ .ci/stop-runner.sh otx-ci-container <github-token>
+   training_extensions$ .ci/docker/stop-runner.sh otx-ci-container <github-token>
    ```
 
    > **_Note_**: If there is an action in progress on the actions-runner which you want to stop, this script will be resulted with an error. To perform force stopping the runner, you can stop the docker container using `docker stop` command on the host machine.
 
 1. Monitor the running runner
    ```bash
-   training_extensions$ .ci/check-runner.sh --help
+   training_extensions$ .ci/docker/check-runner.sh --help
    USAGE: .ci/check-runner.sh <container-name> [Options]
    Options
        -r|--runner         Check runner's log instead of Job one

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -8,6 +8,141 @@ All notable changes to this project will be documented in this file.
 
 ### Enhancements
 
+### Bug fixes
+
+## \[v2.1.0\]
+
+### New features
+
+- Add a flag to enable OV inference on dGPU
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3503>)
+- Add early stopping with warmup. Remove mandatory background label in semantic segmentation task
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3515>)
+- RTMDet-tiny enablement for detection task
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3542>)
+- Add data_format validation and update in OTXDataModule
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3579>)
+- Add torchvision.MaskRCNN
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3504>)
+- Add Semi-SL for Multi-class Classification (EfficientNet-B0)
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3566>)
+- Decoupling mmaction for action classification (MoviNet, X3D)
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3582>)
+- Add Semi-SL Algorithms for mv3-large, effnet-v2, deit-tiny, dino-v2
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3602>)
+- RTMDet-tiny enablement for detection task (export/optimize)
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3564>)
+- Enable ruff & ruff-format into otx/algo/classification/backbones
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3667>)
+- Add TV MaskRCNN Tile Recipe
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3655>)
+- Add rotated det OV recipe
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3687>)
+
+### Enhancements
+
+- Change load_stat_dict to on_load_checkpoint
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3443>)
+- Add try - except to keep running the remaining tests
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3448>)
+- Update instance_segmentation.py to resolve conflict with 2.0.0
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3506>)
+- Update XPU install
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3516>)
+- Sync rgb order between torch and ov inference of action classification task
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3551>)
+- Make Perf test available to load pervious Perf test to skip training stage
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3556>)
+- Reenable e2e classification XAI tests
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3591>)
+- Remove action detection task support
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3605>)
+- Increase readability of pickling error log during HPO & fix minor bug
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3606>)
+- Update RTMDet checkpoint url
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3631>)
+- Refactor Torchvision Model for Classification Semi-SL
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3614>)
+- Add coverage omit mm-related code
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3641>)
+- Add docs semi-sl part
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3640>)
+- Refactor docs design & Add contents
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3645>)
+- Add execution example of auto batch size in docs
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3648>)
+- Add Semi-SL for cls Benchmark Test
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3647>)
+- Move value to device before logging for metric
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3649>)
+- Add .codecov.yaml
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3650>)
+- Update benchmark tool for otx2.1
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3652>)
+- Collect pretrained weight binary files in one place
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3656>)
+- Minimize compiled dependency files
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3653>)
+- Update README & CODEOWNERS
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3659>)
+- Update Engine's docstring & CLI --help outputs
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3658>)
+- Align integration test to exportable code interface update for release branch
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3676>)
+- Refactor exporter for anomaly task and fix a bug with exportable code
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3672>)
+- Update pandas version constraint
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3679>)
+- Include more models to export test into test_otx_e2e
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3678>)
+- Move assigning tasks to Models from Engine to Anomaly Model Classes
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3683>)
+- Refactoring detection modules
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3636>)
+
+### Bug fixes
+
+- Fix conflicts between develop and 2.0.0
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3490>)
+- Fix polygon mask
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3549>)
+- Fix vpm intg test error
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3554>)
+- Fix anomaly
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3557>)
+- Bug fix in Semantic Segmentation + enable DINOV2 export in ONNX
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3569>)
+- Fix some export issues. Remove EXPORTABLE_CODE as export parameter.
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3577>)
+- Fix `load_from_checkpoint` to apply original model's hparams
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3607>)
+- Fix `load_from_checkpoint` args to apply original model's hparams
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3611>)
+- Fix zero-shot `learn` for ov model
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3601>)
+- Various fixes for XAI in 2.1
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3615>)
+- Fix tests to work in a mm-free environment
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3632>)
+- Fix a bug in benchmark code
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3643>)
+- Update exportable code dependency & fix a bug
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3642>)
+- Fix getting wrong shape during resizing
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3644>)
+- Fix detection prediction outputs
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3634>)
+- Fix RTMDet PTQ performance
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3651>)
+- Fix segmentation fault on VPM PTQ
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3654>, <https://github.com/openvinotoolkit/training_extensions/pull/3689>)
+- Fix NNCF MaskRCNN-Eff accuracy drop
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3680>)
+- Fix optimize with Semi-SL data pipeline
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3684>)
+- Fix MaskRCNN SwinT NNCF Accuracy Drop
+  (<https://github.com/openvinotoolkit/training_extensions/pull/3685>)
+
 ## \[v2.0.0\]
 
 > _**NOTES**_

diff --git a/pyproject.toml b/pyproject.toml
@@ -26,7 +26,7 @@ classifiers = [
     "Programming Language :: Python :: 3.11",
 ]
 dependencies = [
-    "datumaro==1.6.1",
+    "datumaro==1.7.0",
     "omegaconf==2.3.0",
     "rich==13.7.1",
     "jsonargparse==4.30.0",
@@ -82,7 +82,7 @@ xpu = [
     "timm",
     "openvino==2024.0",
     "openvino-dev==2024.0",
-    "openvino-model-api==0.2.0",
+    "openvino-model-api==0.2.1",
     "onnx==1.16.1",
     "onnxconverter-common==1.14.0",
     "nncf==2.9.0",

diff --git a/src/otx/__init__.py b/src/otx/__init__.py
@@ -3,7 +3,7 @@
 # Copyright (C) 2023 Intel Corporation
 # SPDX-License-Identifier: Apache-2.0
 
-__version__ = "2.0.0"
+__version__ = "2.2.0rc0"
 
 from otx.core.types import *  # noqa: F403
 

diff --git a/src/otx/algo/anomaly/openvino_model.py b/src/otx/algo/anomaly/openvino_model.py
@@ -11,7 +11,7 @@
 
 from __future__ import annotations
 
-from typing import TYPE_CHECKING, Any, Sequence
+from typing import TYPE_CHECKING, Any, Literal, Sequence
 
 import numpy as np
 import openvino
@@ -26,6 +26,7 @@
 from otx.core.model.anomaly import AnomalyModelInputs
 from otx.core.model.base import OVModel
 from otx.core.types.label import AnomalyLabelInfo
+from otx.core.types.task import OTXTaskType
 
 if TYPE_CHECKING:
     from pathlib import Path
@@ -103,6 +104,11 @@ def __init__(
         use_throughput_mode: bool = True,
         model_api_configuration: dict[str, Any] | None = None,
         metric: MetricCallable = NullMetricCallable,  # Metrics is computed using Anomalib's metric
+        task: Literal[
+            OTXTaskType.ANOMALY_CLASSIFICATION,
+            OTXTaskType.ANOMALY_DETECTION,
+            OTXTaskType.ANOMALY_SEGMENTATION,
+        ] = OTXTaskType.ANOMALY_CLASSIFICATION,
         **kwargs,
     ) -> None:
         super().__init__(
@@ -117,6 +123,7 @@ def __init__(
         metric_names = ["AUROC", "F1Score"]
         self.image_metrics: AnomalibMetricCollection = create_metric_collection(metric_names, prefix="image_")
         self.pixel_metrics: AnomalibMetricCollection = create_metric_collection(metric_names, prefix="pixel_")
+        self.task = task
 
     def _create_model(self) -> Model:
         from model_api.adapters import OpenvinoAdapter, create_core, get_user_config

diff --git a/src/otx/algo/anomaly/padim.py b/src/otx/algo/anomaly/padim.py
@@ -7,13 +7,14 @@
 
 from __future__ import annotations
 
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, Literal
 
 from anomalib.models.image import Padim as AnomalibPadim
 
 from otx.core.model.anomaly import OTXAnomaly
 from otx.core.model.base import OTXModel
 from otx.core.types.label import AnomalyLabelInfo
+from otx.core.types.task import OTXTaskType
 
 if TYPE_CHECKING:
     from lightning.pytorch.utilities.types import STEP_OUTPUT
@@ -30,8 +31,9 @@ class Padim(OTXAnomaly, OTXModel, AnomalibPadim):
         layers (list[str], optional): Feature extractor layers. Defaults to ["layer1", "layer2", "layer3"].
         pre_trained (bool, optional): Pretrained backbone. Defaults to True.
         n_features (int | None, optional): Number of features. Defaults to None.
-        num_classes (int, optional): Anoamly don't use num_classes ,
-            but OTXModel always receives num_classes, so need this.
+        task (Literal[
+                OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION
+            ], optional): Task type of Anomaly Task. Defaults to OTXTaskType.ANOMALY_CLASSIFICATION.
     """
 
     def __init__(
@@ -40,6 +42,11 @@ def __init__(
         layers: list[str] = ["layer1", "layer2", "layer3"],  # noqa: B006
         pre_trained: bool = True,
         n_features: int | None = None,
+        task: Literal[
+            OTXTaskType.ANOMALY_CLASSIFICATION,
+            OTXTaskType.ANOMALY_DETECTION,
+            OTXTaskType.ANOMALY_SEGMENTATION,
+        ] = OTXTaskType.ANOMALY_CLASSIFICATION,
     ) -> None:
         OTXAnomaly.__init__(self)
         OTXModel.__init__(self, label_info=AnomalyLabelInfo())
@@ -50,6 +57,7 @@ def __init__(
             pre_trained=pre_trained,
             n_features=n_features,
         )
+        self.task = task
 
     def configure_optimizers(self) -> tuple[list[Optimizer], list[Optimizer]] | None:
         """PADIM doesn't require optimization, therefore returns no optimizers."""

diff --git a/src/otx/algo/anomaly/stfpm.py b/src/otx/algo/anomaly/stfpm.py
@@ -7,13 +7,14 @@
 
 from __future__ import annotations
 
-from typing import TYPE_CHECKING, Sequence
+from typing import TYPE_CHECKING, Literal, Sequence
 
 from anomalib.models.image.stfpm import Stfpm as AnomalibStfpm
 
 from otx.core.model.anomaly import OTXAnomaly
 from otx.core.model.base import OTXModel
 from otx.core.types.label import AnomalyLabelInfo
+from otx.core.types.task import OTXTaskType
 
 if TYPE_CHECKING:
     from lightning.pytorch.utilities.types import STEP_OUTPUT
@@ -28,14 +29,20 @@ class Stfpm(OTXAnomaly, OTXModel, AnomalibStfpm):
     Args:
         layers (Sequence[str]): Feature extractor layers.
         backbone (str, optional): Feature extractor backbone. Defaults to "resnet18".
-        num_classes (int, optional): Anoamly don't use num_classes ,
-            but OTXModel always receives num_classes, so need this.
+        task (Literal[
+                OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION
+            ], optional): Task type of Anomaly Task. Defaults to OTXTaskType.ANOMALY_CLASSIFICATION.
     """
 
     def __init__(
         self,
         layers: Sequence[str] = ["layer1", "layer2", "layer3"],
         backbone: str = "resnet18",
+        task: Literal[
+            OTXTaskType.ANOMALY_CLASSIFICATION,
+            OTXTaskType.ANOMALY_DETECTION,
+            OTXTaskType.ANOMALY_SEGMENTATION,
+        ] = OTXTaskType.ANOMALY_CLASSIFICATION,
         **kwargs,
     ) -> None:
         OTXAnomaly.__init__(self)
@@ -45,6 +52,7 @@ def __init__(
             backbone=backbone,
             layers=layers,
         )
+        self.task = task
 
     @property
     def trainable_model(self) -> str:

diff --git a/src/otx/algo/common/backbones/__init__.py b/src/otx/algo/common/backbones/__init__.py
@@ -0,0 +1,11 @@
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+"""Custom backbone implementations."""
+
+from .cspnext import CSPNeXt
+from .pytorchcv_backbones import build_model_including_pytorchcv
+from .resnet import ResNet
+from .resnext import ResNeXt
+
+__all__ = ["CSPNeXt", "build_model_including_pytorchcv", "ResNet", "ResNeXt"]