Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CodeCamp #106: Support SOLOv2 deployment with OpenVINO #1521

Merged
merged 43 commits into from
Apr 26, 2023
Merged

CodeCamp #106: Support SOLOv2 deployment with OpenVINO #1521

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
67a3ad0
support solov2 openvino
Li-Hongda Dec 12, 2022
341f7e5
fix lint
Li-Hongda Dec 12, 2022
eff6814
Update benchmark.md
Li-Hongda Dec 16, 2022
11a44ac
Update __init__.py
Li-Hongda Dec 16, 2022
2b544fc
fix names and delete redundant codes
Li-Hongda Dec 16, 2022
238f1f4
Merge branch 'Li-Hongda/CodeCamp#106' of https://github.com/Li-Hongda…
Li-Hongda Dec 16, 2022
e941248
fix names and redundant codes
Li-Hongda Dec 16, 2022
6c7465e
fix indent
Li-Hongda Dec 16, 2022
862b29a
fix lint
Li-Hongda Dec 16, 2022
a21ae86
fix lint
Li-Hongda Dec 16, 2022
084c275
Merge branch 'dev-1.x' of https://github.com/open-mmlab/mmdeploy into…
Li-Hongda Dec 19, 2022
e5b6943
delete 'ctx' arguments
Li-Hongda Dec 19, 2022
84a8c93
[Fix]fix lint
Li-Hongda Dec 19, 2022
e9e9707
修改： mmdeploy/codebase/mmdet/models/dense_heads/solov2_head.py
Li-Hongda Dec 25, 2022
108f747
Merge branch 'dev-1.x' of https://github.com/open-mmlab/mmdeploy into…
Li-Hongda Dec 25, 2022
549ae1c
Delete matrix_nms.py
Li-Hongda Dec 29, 2022
55c0375
Update __init__.py
Li-Hongda Dec 29, 2022
ca68207
Update linspace.py
Li-Hongda Dec 29, 2022
f36f21c
Update solov2_head.py
Li-Hongda Dec 29, 2022
efda8b7
Update solov2_head.py
Li-Hongda Dec 29, 2022
c78a1f6
update
Li-Hongda Mar 31, 2023
7033c26
Merge branch 'Li-Hongda/CodeCamp#106' of https://github.com/Li-Hongda…
Li-Hongda Mar 31, 2023
ad56660
Merge branch 'dev-1.x' of https://github.com/open-mmlab/mmdeploy into…
Li-Hongda Mar 31, 2023
f28a647
fix num_groups
Li-Hongda Apr 1, 2023
4385adc
update docs
Li-Hongda Apr 2, 2023
a7fae9f
Merge branch 'main' of https://github.com/open-mmlab/mmdeploy into Li…
Li-Hongda Apr 10, 2023
f5b3e26
Merge branch 'main' of https://github.com/open-mmlab/mmdeploy into Li…
Li-Hongda Apr 12, 2023
1dea0bd
add end2endmodel for solo
Li-Hongda Apr 12, 2023
67383d5
add end2endmodel for solo
Li-Hongda Apr 12, 2023
2d61799
[Feature] Support End2EndModelSOLO
Li-Hongda Apr 12, 2023
e94aa1f
Update object_detection_model.py
Li-Hongda Apr 12, 2023
9bce17b
add model_cfg to End2EndModel
Li-Hongda Apr 13, 2023
1bd60c2
Merge branch 'Li-Hongda/CodeCamp#106' of https://github.com/Li-Hongda…
Li-Hongda Apr 13, 2023
01c6488
add model_cfg to End2EndModel
Li-Hongda Apr 13, 2023
27bca49
Fix bugs for End2EndModel
Li-Hongda Apr 13, 2023
3844cbb
fix bugs for End2EndModel
Li-Hongda Apr 13, 2023
3aa6efe
update docs.
Li-Hongda Apr 14, 2023
a17d23d
update conmment
Li-Hongda Apr 14, 2023
af90a8d
更新 benchmark.md
Li-Hongda Apr 14, 2023
f3dbdfe
update regression tests
Li-Hongda Apr 17, 2023
62c31a4
Merge branches 'Li-Hongda/CodeCamp#106' and 'Li-Hongda/CodeCamp#106' …
Li-Hongda Apr 17, 2023
0668f98
fix errors in docs
Li-Hongda Apr 17, 2023
c3ea756
Merge branch 'main' of https://github.com/open-mmlab/mmdeploy into Li…
Li-Hongda Apr 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions docs/en/03-benchmark/benchmark.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">81.18</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
</tr>
<tr>
<td align="center">top-5</td>
Expand All @@ -623,6 +624,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">95.61</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
</tr>
</tbody>
</table>
Expand Down Expand Up @@ -907,20 +909,35 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo">SOLO</a></td>
<td align="center" rowspan="2">Instance Segmentation</td>
<td align="center" rowspan="2">COCO2017</td>
<td align="center"><a href="https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo/solo_r50_fpn_1x_coco.py">SOLO</a></td>
<td align="center">Instance Segmentation</td>
<td align="center">COCO2017</td>
<td align="center">mask AP</td>
<td align="center">33.1</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">32.7</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">32.7</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solov2/solov2_r50_fpn_1x_coco.py">SOLOv2</a></td>
<td align="center">Instance Segmentation</td>
<td align="center">COCO2017</td>
<td align="center">mask AP</td>
<td align="center">34.8</td>
<td align="center">-</td>
<td align="center">34.5</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">34.5</td>
</tr>
</tbody>
</table>
</div>
Expand Down
3 changes: 2 additions & 1 deletion docs/en/03-benchmark/supported_models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ The table below lists the models that are guaranteed to be exportable to other b
| [RepPoints](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/reppoints) | MMDetection | N | N | Y | N | ? | Y | N | N |
| [DETR](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/detr) | MMDetection | N | Y | Y | N | ? | N | N | N |
| [CenterNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/centernet) | MMDetection | N | Y | Y | N | ? | N | N | N |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | MMDetection | N | N | N | N | N | Y | N | N |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | MMDetection | N | Y | N | N | N | Y | N | N |
| [SOLOv2](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solov2) | MMDetection | N | Y | N | N | N | Y | N | N |
| [ResNet](https://github.com/open-mmlab/mmclassification/tree/1.x/configs/resnet) | MMClassification | Y | Y | Y | Y | Y | Y | Y | Y |
| [ResNeXt](https://github.com/open-mmlab/mmclassification/tree/1.x/configs/resnext) | MMClassification | Y | Y | Y | Y | Y | Y | Y | Y |
| [SE-ResNet](https://github.com/open-mmlab/mmclassification/tree/1.x/configs/seresnet) | MMClassification | Y | Y | Y | Y | Y | Y | Y | Y |
Expand Down
3 changes: 2 additions & 1 deletion docs/en/04-supported-codebases/mmdet.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,4 +210,5 @@ Besides python API, mmdeploy SDK also provides other FFI (Foreign Function Inter
| [Cascade Mask R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/cascade_rcnn) | Instance Segmentation | Y | N | N | N | Y |
| [Mask R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/mask_rcnn) | Instance Segmentation | Y | Y | N | N | Y |
| [Swin Transformer](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/swin) | Instance Segmentation | Y | Y | N | N | N |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | Instance Segmentation | N | N | N | N | Y |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | Instance Segmentation | Y | N | N | N | Y |
| [SOLOv2](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solov2) | Instance Segmentation | Y | N | N | N | Y |
25 changes: 20 additions & 5 deletions docs/zh_cn/03-benchmark/benchmark.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -633,7 +633,7 @@ GPU: ncnn, TensorRT, PPLNN
<th align="center" colspan="3">TensorRT</th>
<th align="center">PPLNN</th>
<th align="center">Ascend</th>
<th algin="center">OpenVINO</th>
<th align="center">OpenVINO</th>
</tr>
</thead>
<tbody>
Expand Down Expand Up @@ -902,20 +902,35 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo">SOLO</a></td>
<td align="center" rowspan="2">Instance Segmentation</td>
<td align="center" rowspan="2">COCO2017</td>
<td align="center"><a href="https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo/solo_r50_fpn_1x_coco.py">SOLO</a></td>
<td align="center">Instance Segmentation</td>
<td align="center">COCO2017</td>
<td align="center">mask AP</td>
<td align="center">33.1</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">32.7</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">32.7</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solov2/solov2_r50_fpn_1x_coco.py">SOLOv2</a></td>
<td align="center">Instance Segmentation</td>
<td align="center">COCO2017</td>
<td align="center">mask AP</td>
<td align="center">34.8</td>
<td align="center">-</td>
<td align="center">34.5</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">34.5</td>
</tr>
</tbody>
</table>
</div>
Expand Down
3 changes: 2 additions & 1 deletion docs/zh_cn/03-benchmark/supported_models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
| [RepPoints](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/reppoints) | MMDetection | N | N | Y | N | ? | Y | N | N |
| [DETR](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/detr) | MMDetection | N | Y | Y | N | ? | N | N | N |
| [CenterNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/centernet) | MMDetection | N | Y | Y | N | ? | N | N | N |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | MMDetection | N | N | N | N | N | Y | N | N |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | MMDetection | N | Y | N | N | N | Y | N | N |
| [SOLOv2](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solov2) | MMDetection | N | Y | N | N | N | Y | N | N |
| [ResNet](https://github.com/open-mmlab/mmclassification/tree/1.x/configs/resnet) | MMClassification | Y | Y | Y | Y | Y | Y | Y | Y |
| [ResNeXt](https://github.com/open-mmlab/mmclassification/tree/1.x/configs/resnext) | MMClassification | Y | Y | Y | Y | Y | Y | Y | Y |
| [SE-ResNet](https://github.com/open-mmlab/mmclassification/tree/1.x/configs/seresnet) | MMClassification | Y | Y | Y | Y | Y | Y | Y | Y |
Expand Down
45 changes: 23 additions & 22 deletions docs/zh_cn/04-supported-codebases/mmdet.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,25 +192,26 @@ cv2.imwrite('output_detection.png', img)

## 模型支持列表

| Model | Task | OnnxRuntime | TensorRT | ncnn | PPLNN | OpenVINO |
| :-------------------------------------------------------------------------------------------: | :-------------------: | :---------: | :------: | :--: | :---: | :------: |
| [ATSS](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/atss) | ObjectDetection | Y | Y | N | N | Y |
| [FCOS](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/fcos) | ObjectDetection | Y | Y | Y | N | Y |
| [FoveaBox](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/foveabox) | ObjectDetection | Y | N | N | N | Y |
| [FSAF](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/fsaf) | ObjectDetection | Y | Y | Y | Y | Y |
| [RetinaNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/retinanet) | ObjectDetection | Y | Y | Y | Y | Y |
| [SSD](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/ssd) | ObjectDetection | Y | Y | Y | N | Y |
| [VFNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/vfnet) | ObjectDetection | N | N | N | N | Y |
| [YOLOv3](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/yolo) | ObjectDetection | Y | Y | Y | N | Y |
| [YOLOX](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/yolox) | ObjectDetection | Y | Y | Y | N | Y |
| [Cascade R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/cascade_rcnn) | ObjectDetection | Y | Y | N | Y | Y |
| [Faster R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/faster_rcnn) | ObjectDetection | Y | Y | Y | Y | Y |
| [Faster R-CNN + DCN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/faster_rcnn) | ObjectDetection | Y | Y | Y | Y | Y |
| [GFL](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/gfl) | ObjectDetection | Y | Y | N | ? | Y |
| [RepPoints](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/reppoints) | ObjectDetection | N | Y | N | ? | Y |
| [DETR](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/detr) | ObjectDetection | Y | Y | N | ? | Y |
| [CenterNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/centernet) | Object Detection | Y | Y | N | ? | ? |
| [Cascade Mask R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/cascade_rcnn) | InstanceSegmentation | Y | N | N | N | Y |
| [Mask R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/mask_rcnn) | InstanceSegmentation | Y | Y | N | N | Y |
| [Swin Transformer](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/swin) | InstanceSegmentation | Y | Y | N | N | N |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | Instance Segmentation | N | N | N | N | Y |
| Model | Task | OnnxRuntime | TensorRT | ncnn | PPLNN | OpenVINO |
| :-------------------------------------------------------------------------------------------: | :------------------: | :---------: | :------: | :--: | :---: | :------: |
| [ATSS](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/atss) | ObjectDetection | Y | Y | N | N | Y |
| [FCOS](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/fcos) | ObjectDetection | Y | Y | Y | N | Y |
| [FoveaBox](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/foveabox) | ObjectDetection | Y | N | N | N | Y |
| [FSAF](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/fsaf) | ObjectDetection | Y | Y | Y | Y | Y |
| [RetinaNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/retinanet) | ObjectDetection | Y | Y | Y | Y | Y |
| [SSD](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/ssd) | ObjectDetection | Y | Y | Y | N | Y |
| [VFNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/vfnet) | ObjectDetection | N | N | N | N | Y |
| [YOLOv3](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/yolo) | ObjectDetection | Y | Y | Y | N | Y |
| [YOLOX](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/yolox) | ObjectDetection | Y | Y | Y | N | Y |
| [Cascade R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/cascade_rcnn) | ObjectDetection | Y | Y | N | Y | Y |
| [Faster R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/faster_rcnn) | ObjectDetection | Y | Y | Y | Y | Y |
| [Faster R-CNN + DCN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/faster_rcnn) | ObjectDetection | Y | Y | Y | Y | Y |
| [GFL](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/gfl) | ObjectDetection | Y | Y | N | ? | Y |
| [RepPoints](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/reppoints) | ObjectDetection | N | Y | N | ? | Y |
| [DETR](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/detr) | ObjectDetection | Y | Y | N | ? | Y |
| [CenterNet](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/centernet) | Object Detection | Y | Y | N | ? | ? |
| [Cascade Mask R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/cascade_rcnn) | InstanceSegmentation | Y | N | N | N | Y |
| [Mask R-CNN](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/mask_rcnn) | InstanceSegmentation | Y | Y | N | N | Y |
| [Swin Transformer](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/swin) | InstanceSegmentation | Y | Y | N | N | N |
| [SOLO](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solo) | InstanceSegmentation | Y | N | N | N | Y |
| [SOLOv2](https://github.com/open-mmlab/mmdetection/tree/3.x/configs/solov2) | InstanceSegmentation | Y | N | N | N | Y |
6 changes: 6 additions & 0 deletions mmdeploy/codebase/mmdet/deploy/object_detection_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,13 @@ def __init__(self,
backend_files: Sequence[str],
device: str,
deploy_cfg: Union[str, Config],
model_cfg: Optional[Union[str, Config]] = None,
data_preprocessor: Optional[Union[dict, nn.Module]] = None,
**kwargs):
super().__init__(
deploy_cfg=deploy_cfg, data_preprocessor=data_preprocessor)
self.deploy_cfg = deploy_cfg
self.model_cfg = model_cfg
self.device = device
self._init_wrapper(
backend=backend, backend_files=backend_files, device=device)
Expand Down Expand Up @@ -195,6 +197,8 @@ def forward(self,
img_metas = [data_sample.metainfo for data_sample in data_samples]
results = []
rescale = kwargs.get('rescale', True)
model_type = self.model_cfg.model.type if \
self.model_cfg is not None else None
for i in range(batch_size):
dets, labels = batch_dets[i], batch_labels[i]
result = InstanceData()
Expand Down Expand Up @@ -234,6 +238,8 @@ def forward(self,

result.scores = scores
result.bboxes = bboxes
if model_type in ['SOLO', 'SOLOv2']:
result.bboxes = bboxes.new_zeros(bboxes.shape)
result.labels = labels

if batch_masks is not None:
Expand Down
1 change: 1 addition & 0 deletions mmdeploy/codebase/mmdet/models/dense_heads/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@
from . import rtmdet_head # noqa: F401,F403
from . import rtmdet_ins_head # noqa: F401,F403
from . import solo_head # noqa: F401,F403
from . import solov2_head # noqa: F401,F403
from . import yolo_head # noqa: F401,F403
from . import yolox_head # noqa: F401,F403
17 changes: 9 additions & 8 deletions mmdeploy/codebase/mmdet/models/dense_heads/solo_head.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,15 @@


@FUNCTION_REWRITER.register_rewriter(
'mmdet.models.dense_heads.SOLOHead.predict_by_feat', backend='openvino')
def solohead__predict_by_feat__openvino(self,
mlvl_mask_preds: List[Tensor],
mlvl_cls_scores: List[Tensor],
batch_img_metas: List[Dict],
cfg: OptConfigType = None,
**kwargs):
"""Rewrite `predict_by_feat` of `SOLOHead` for openvino backend."""
func_name='mmdet.models.dense_heads.solo_head.'
'SOLOHead.predict_by_feat')
def solohead__predict_by_feat(self,
mlvl_mask_preds: List[Tensor],
mlvl_cls_scores: List[Tensor],
batch_img_metas: List[Dict],
cfg: OptConfigType = None,
**kwargs):
"""Rewrite `predict_by_feat` of `SOLOHead` for default backend."""

ctx = FUNCTION_REWRITER.get_context()
batch_size = mlvl_cls_scores[0].size(0)
Expand Down
Loading