Recipe refinement (det, iseg, r-det, sseg, vpm, zsl)

[sungchul2]

Summary

This PR includes:

• Create base recipe for each task

before

after

recipe/_base_ ├── data │   ├── mmaction_base.yaml │   ├── mmdet_base.yaml │   ├── mmpretrain_base.yaml │   ├── mmseg_base.yaml │   ├── torchvision_base.yaml │   ├── torchvision_semisl.yaml ├── test.yaml └── train.yaml

recipe/_base_ ├── data │   ├── detection.yaml * │   ├── instance_segmentation.yaml * │   ├── mmaction_base.yaml │   ├── mmdet_base.yaml │   ├── mmpretrain_base.yaml │   ├── mmseg_base.yaml │   ├── rotated_detection.yaml * │   ├── semantic_segmentation.yaml * │   ├── torchvision_base.yaml │   ├── torchvision_semisl.yaml │   ├── visual_prompting.yaml * │   └── zero_shot_visual_prompting.yaml * ├── test.yaml └── train.yaml

• Remove configs which have default value
• Performance check w/ medium dataset for reproducibility
  • all benchmark tests proceeded with medium dataset, single trial, and seed = 0, these results are just for checking trend
  • before : 2adf5b1 | after : 7849186
  • detection

    model		test/f1-score	export/f1-score
    atss_mobilenetv2	before	0.73	0.73
    	after	0.73	0.73
    atss_resnext101	before	0.75	0.76
    	after	0.75	0.76
    rtmdet_tiny	before	0.71	0.73
    	after	0.74	0.71
    ssd_mobilenetv2	before	0.62	0.62
    	after	0.62	0.62
    yolox_l	before	0.73	0.72
    	after	0.73	0.72
    yolox_s	before	0.71	0.70
    	after	0.71	0.70
    yolox_tiny	before	0.73	0.74
    	after	0.73	0.74
    yolox_x	before	0.74	0.71
    	after	0.74	0.71

  • instance segmentation
    • maskrcnn_r50_tv randomly has an overflow issue of label prediction of OVIR in develop

      model		test/f1-score	export/f1-score
      maskrcnn_efficientnetb2b	before	0.57	0.57
      	after	0.57	0.57
      maskrcnn_r50	before	0.59	0.60
      	after	0.59	0.59
      maskrcnn_r50_tv	before	0.69	-
      	after	0.69	0.69
      maskrcnn_swint	before	0.70	0.69
      	after	0.68	0.68
      rtmdet_inst_tiny	before	0.60	0.61
      	after	0.57	0.57

  • rotated detection -> no benchmark test
  • semantic segmentation

    model		test/Dice	export/Dice
    dino_v2	before	0.90	0.90
    	after	0.90	0.90
    litehrnet_18	before	0.91	0.92
    	after	0.90	0.89
    litehrnet_s	before	0.89	0.88
    	after	0.89	0.89
    litehrnet_x	before	0.89	0.87
    	after	0.90	0.88
    segnext_b	before	0.90	0.90
    	after	0.91	0.89
    segnext_s	before	0.90	0.90
    	after	0.91	0.91
    segnext_t	before	0.90	0.90
    	after	0.90	0.89

  • visual prompting

    model		test/dice	export/dice
    sam_tiny_vit	before	0.91	0.99
    	after	0.91	0.99
    sam_vit_b	before	0.93	0.99
    	after	0.93	1.00
    sam_tiny_vit (zsl)	before	0.60	0.60
    	after	0.60	0.60
    sam_vit_b (zsl)	before	0.33	0.33
    	after	0.33	0.33

How to test

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have ran e2e tests and there is no issues.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.54%. Comparing base (f9808c5) to head (2c03cb0).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3712      +/-   ##
===========================================
- Coverage    80.55%   80.54%   -0.01%     
===========================================
  Files          254      254              
  Lines        25611    25610       -1     
===========================================
- Hits         20631    20628       -3     
- Misses        4980     4982       +2     

Flag	Coverage Δ
py310	80.54% <ø> (+<0.01%)	⬆️
py311	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[harimkang]

In my opinion, this is a pretty big change, and one that was discussed a lot during the config design discussions for OTX 2.0. If a change is needed, it needs more discussion.

[wonjuleee]

Great thank you for the contribution!