[Hackthon_4th 242] Support en_ppstructure_mobile_v2.0_SLANet

[thunder95]

PR types(PR类型)

Model

Description

完成ppstructure中表格识别模型en_ppstructure_mobile_v2.0_SLANet算法前后处理，及精度对齐，开发Python部署示例和C++部署示例

[thunder95]

paddle inference以及onnxruntime测试验证通过
openvino报错：　[INFO] fastdeploy/runtime/runtime.cc(279)::CreateOpenVINOBackend Runtime initialized with Backend::OPENVINO in Device::CPU.
terminate called after throwing an instance of 'ov::Exception'
what(): Check 'creator_it != CREATORS_MAP.end()' failed at src/frontends/paddle/src/frontend.cpp:46:
FrontEnd API failed with OpConversionFailure: :
No creator found for one_hot_v2 node.

TRT报错：
[ERROR] fastdeploy/runtime/backends/tensorrt/trt_backend.cc(695)::CreateTrtEngineFromOnnx Failed to parse ONNX model by TensorRT.
[ERROR] fastdeploy/runtime/backends/tensorrt/trt_backend.cc(298)::InitFromOnnx Failed to create tensorrt engine.
[ERROR] fastdeploy/runtime/runtime.cc(300)::CreateTrtBackend Failed to initialize TensorRT backend.

验证, 性能benchmark, 结果输出无差异：
======= Model Info =======
model_name: en_ppstructure_mobile_v2.0_SLANet_infer
profile_mode: runtime
include_h2d_d2h: false

======= Backend Info =======
warmup: 50
repeats: 200
device: gpu
device_id: 0
use_fp16: false
backend: paddle
collect_memory_info: false

[WARNING] fastdeploy/fastdeploy_model.cc(48)::IsSupported In benchmark mode, we don't check to see if the backend [Backend::PDINFER] is supported for current model!
[INFO] fastdeploy/runtime/runtime.cc(266)::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::GPU.
[INFO] fastdeploy/runtime/backends/paddle/paddle_backend.cc(309)::Infer Running profiling for Runtime without H2D and D2H, Repeats: 200, Warmup: 50
=============== Test Table Result diff =================
PPOCR Table structure has no diff
table boxes num matched with expected: 320
box coords, max_diff: 0, , total diff: 0, average diff: 0
[INFO] fastdeploy/runtime/backends/paddle/paddle_backend.cc(309)::Infer Running profiling for Runtime without H2D and D2H, Repeats: 200, Warmup: 50
Runtime(ms): 333.77ms.

模型串联:文本检测＋文本识别＋表格识别：
预期(ppocr输出):

Methods	R	P	F	FPS
SegLink [26]	70.0	86.0	77.0	8.9
PixelLink [4]	73.2	83.0	77.8	-
TextSnake [18]	73.9	83.2	78.3	1.1
TextField [37]	75.9	87.4	81.3	5.2
MSR[38]	76.7	87.4	81.7	-
FTSN[3]	77.1	87.6	82.0	-
LSE[30]	81.7	84.2	82.9	-
CRAFT [2]	78.2	88.2	82.9	8.6
MCN [16]	79	88.	83	-
ATRR[35]	82.1	85.2	83.6	-
PAN [34]	83.8	84.4	84.1	30.2
DB[12]	79.2	91.5	84.9	32.0
DRRG [41]	82.30	88.05	85.08	-
Ours (SynText)	80.68	85.40	82.97	12.68
Ours (MLT-17)	84.54	86.62	85.57	12.31

实际(fastdeploy输出):

Methods	R	P	F	FPS
SegLink [26]	70.0	86.0	77.0	8.9
PixelLink [4]	73.2	83.0	77.8	-
TextSnake [18]	73.9	83.2	78.3	1.1
TextField [37]	75.9	87.4	81.3	5.2
MSR[38]	76.7	87.4	81.7	-
FTSN[3]	77.1	87.6	82.0	-
LSE[30]	81.7	84.2	82.9	-
CRAFT [2]	78.2	88.2	82.9	8.6
MCN [16]	79	88.	83	-
ATRR[35]	82.1	85.2	83.6	-
PAN [34]	83.8	84.4	84.1	30.2
DB[12]	79.2	91.5	84.9	32.0
DRRG [41]	82.30	88.05	85.08	-
Ours (SynText)	80.68	85.40	82.97	12.68
Ours (MLT-17)	84.54	86.62	85.57	12.31

结论：　输出结果一致

[DefTruth]

LGTM

[DefTruth]

ppstructure相关的模型，需要使用ppstructure进行区分。

ppocr_table.cc -> ppstructurev2_table.cc 
ppocr_table.h -> ppstructurev2_table.h
table.cc -> structurev2_table.cc 
table.h -> structurev2_table.h
table_preprocessor.cc ->  structurev2_table_preprocessor.cc
table_preprocessor.h ->  structurev2_table_preprocessor.h
table_postprocessor.cc ->  structurev2_table_postprocessor.cc
table_postprocessor.h ->  structurev2_table_postprocessor.h

命名空间约束，保持现状即可，但需要规范相关类名。如下：

fastdeploy::vision::ocr::Table -> fastdeploy::vision::ocr::StructureV2Table; 
fastdeploy::vision::ocr::TablePostprocessor -> fastdeploy::vision::ocr::StructureV2TablePostprocessor; 
fastdeploy::vision::ocr::TablePreprocessor -> fastdeploy::vision::ocr::StructureV2TablePreprocessor; 

pipeline命名需要修改成：

fastdeploy::pipeline::PPOCRTable -> fastdeploy::pipeline::PPStructureV2Table; 

[DefTruth]

同理，Python 按照类似规范进行修改

[thunder95]

@DefTruth 已修改完成

[thunder95]

支持使用英文模型和英文表格字典推理结果：
./infer_ppstructurev2_table ./ch_PP-OCRv3_det_infer ./ch_PP-OCRv3_rec_infer ./en_ppstructure_mobile_v2.0_SLANet_infer ./ppocr_keys_v1.txt ./table_structure_dict.txt ./en_table.jpg 0

支持使用中文模型和中文表格字典推理结果：

./infer_ppstructurev2_table ./ch_PP-OCRv3_det_infer ./ch_PP-OCRv3_rec_infer ./ch_ppstructure_mobile_v2.0_SLANet_infer ./ppocr_keys_v1.txt ./table_structure_dict_ch.txt ./ch_table.png 0

[DefTruth]

@thunder95 LGTM !

[gl94]

确实 SLANet不支持用openvino推理，可能是了GRU的原因？

[TylunasLi]

OCRResult中相应结果需要提供 python binding

diff --git a/fastdeploy/vision/vision_pybind.cc b/fastdeploy/vision/vision_pybind.cc
index 664e6cc1..c1f26fbd 100644
--- a/fastdeploy/vision/vision_pybind.cc
+++ b/fastdeploy/vision/vision_pybind.cc
@@ -160,6 +160,9 @@ void BindVision(pybind11::module& m) {
       .def_readwrite("rec_scores", &vision::OCRResult::rec_scores)
       .def_readwrite("cls_scores", &vision::OCRResult::cls_scores)
       .def_readwrite("cls_labels", &vision::OCRResult::cls_labels)
+      .def_readwrite("table_boxes", &vision::OCRResult::table_boxes)
+      .def_readwrite("table_structure", &vision::OCRResult::table_structure)
+      .def_readwrite("table_html", &vision::OCRResult::table_html)
       .def("__repr__", &vision::OCRResult::Str)
       .def("__str__", &vision::OCRResult::Str);