[Bug] GatherND shape conversion from ONNX is inaccurate #7379
Comments
|
Hi, OpenVINO does support varieties of models and topologies. However, it is limited. If you noticed, your model topology is not listed in that documentation. Hence, it is not supported and issues are expected. |
Iffa-Intel
added
category: MO
|
Thank you. Even if it's not a "bug," the behavior is clearly strange. I just wanted to report that the behavior is not correct. |
|
Not sure if you had noticed this or not, you will need to use the --input_shape parameter. Its shape is defined as a comma-separated list of integer numbers enclosed in parentheses or square brackets, for example [1,3,227,227] or (1,227,227,3) where the order of dimensions depends on the framework input layout of the model. For example, [N,C,H,W] is used for Caffe* models and [N,H,W,C] for TensorFlow* models. Generally, Model Optimizer performs necessary transformations to convert the shape to the layout required by Inference Engine(N,C,H,W). The shape should not contain undefined dimensions (? or -1) and should fit the dimensions defined in the input operation of the graph. Notes: |
Yes. There are no undefined dimensions. Also, the model is NCHW. |
|
Hi @PINTO0309 Could you double check the Regards, |
|
@jgespino Thank you for your reply. 😄 I tried inference using ONNX alone without https://github.com/ibaiGorordo/ONNX-HITNET-Stereo-Depth-estimation I issued an issue because there does not seem to be any problem with the structure of the model itself. However, I am not familiar with how the internal workings of OpenVINO work, so I don't know what else I can investigate and provide you guys with information. The program to check the operation of ONNX is @ibaiGorordo created, but I generated the ONNX model and gave it to him. Although not the entire program, below is a portion of the program for inputting two still images into onnx runtime and getting the depth estimation results. import cv2
from hitnet import HitNet, ModelType, draw_disparity, draw_depth, CameraConfig, load_img
import numpy as np
from imread_from_url import imread_from_url
if __name__ == '__main__':
# Select model type
# model_type = ModelType.middlebury
# model_type = ModelType.flyingthings
model_type = ModelType.eth3d
if model_type == ModelType.middlebury:
model_path = "models/middlebury_d400/saved_model_480x640/model_float32.onnx"
elif model_type == ModelType.flyingthings:
model_path = "models/flyingthings_finalpass_xl/saved_model_480x640/model_float32.onnx"
elif model_type == ModelType.eth3d:
model_path = "models/eth3d/saved_model_480x640/model_float32.onnx"
# Initialize model
hitnet_depth = HitNet(model_path, model_type)
# Load images
left_img = imread_from_url("https://vision.middlebury.edu/stereo/data/scenes2003/newdata/cones/im2.png")
right_img = imread_from_url("https://vision.middlebury.edu/stereo/data/scenes2003/newdata/cones/im6.png")
# Estimate the depth
disparity_map = hitnet_depth(left_img, right_img)
color_disparity = draw_disparity(disparity_map)
color_disparity = cv2.resize(color_disparity, (left_img.shape[1],left_img.shape[0]))
cobined_image = np.hstack((left_img, right_img, color_disparity))
cv2.imwrite("out.jpg", cobined_image)
cv2.namedWindow("Estimated disparity", cv2.WINDOW_NORMAL)
cv2.imshow("Estimated disparity", cobined_image)
cv2.waitKey(0)
cv2.destroyAllWindows() |
|
@PINTO0309 Let me check with the development team for additional insight. Regards, Ref. 65974 |
|
Apologies for the delay, the development team has implemented a new GatherND_8 operation and should be available in the next release. You could also try building from master branch if you need it sooner. Regards, |
1. System information (version)
OpenVINO=> 2021.4.0-3839
Operating System / Platform => Ubuntu 20.04 x86_64
Compiler => GLIBC 2.31, g++ 9.3.0
Problem classification => Model Conversion
Framework: ONNX
opset=12(TensorFlow)Model name: HITNET
ONNX, tflite, TensorFlow sample
2. Detailed description
Converting a
GatherNDfrom ONNX withbatch_dims=3set does not produce the expected output shape. Therefore, in the subsequent operation transformation operation, the dimension transformation will be invalid and the optimizer will Abort.For example, the tool works as follows.
However, the originally expected behavior is as follows.
Log message. (The
@@@@part was output by adding debug prints to the Model Optimizer logic by myself.)The operational part of ONNX where the problem occurs is shown in the figure below.
https://docs.openvinotoolkit.org/latest/openvino_docs_ops_movement_GatherND_5.html
3. Steps to reproduce
${INTEL_OPENVINO_DIR}/deployment_tools/model_optimizer/mo.py \ --input_model model_float32_opt.onnx \ --data_type FP32 \ --output_dir openvino/FP32 \ --log_level=DEBUGoption: Original HITNET model URL
4. Issue submission checklist
The text was updated successfully, but these errors were encountered: