Multiple BBoxes with SSD

[marvision-ai]

Hello Jk,

I have confirmed SSD works with Xavier and is running at 130FPS using async!! Amazing.

Question: When running the SSD demo, are using NMS? It seems when I run it with a live camera feed, there are multiple boxes around each object. This isnt the same issue as when using a single image.

Thanks!

[jkjung-avt]

Thanks for letting me know your test result on Jetson AGX Xavier.

As to NMS, yes. The TRT SSD engine uses the "NMS_TRT" plugin (provided by NVIDIA) for that. Please refer to ssd/build_engine.py for more info.

The inference code is exactly the same for live camera feed and image file. If you'd like to try "more aggressive" NMS, you could try to lower nmsThreshold value, to say 0.3, and rebuild the TRT engine.

[marvision-ai]

Thanks for the recommendation!

[marvision-ai]

Hi @jkjung-avt I would like to reopen this issue...

I still seem to get multiple boxes. Is this because of the multithreading? Is the Xavier too fast?
Just for information, this is using the regular trt_ssd.py... Not the async version.
See the image below:

[jkjung-avt]

@mbufi, This looks like "NMS" does not properly remove duplicated detections. Did you set nmsThreshold too high? Could you set it back to 0.6 and see if the problem persists?

[marvision-ai]

Hi @jkjung-avt this is actually with the default values from the GitHub. I just reccloned the repo and ran the image object detection on the Huskies with an Xavier and got this.

[jkjung-avt]

I don't see such a problem on either my Jetson Nano DevKit (TensorRT 5 & 6) or x86 PC (TensorRT 5.1.5.0). I still think this is a problem related to NMS.

Which version of TensorRT are you using? Do you have a different platform (or a different version of TensorRT) for comparison?

[marvision-ai]

That is so weird! I will have to try it on multiple jetsons (I have tx2, nano, and Xavier) and let you know the results.

I am using jetpack 4.2.2 which I think is tensorrt 5.1 (off the top of my head).

[jkjung-avt]

I also tested the code with JetPack-4.2.2 (TensorRT 5.1.6.0) before and did not see this problem.

This might be a silly thing to ask. But did you test with the original huskies picture (without bounding boxes)? Make sure you are not using this picture for testing: huskies.png.

[marvision-ai]

Yup! This all started because I tested with live webcam and it was producing multiple bounding boxes. I will experiment more and let you know the results. Thanks jk!

[marvision-ai]

working as intended. Not sure why at first it did not... flashing the jetson and reinstalling all libraries fixed issue...

[Hrayo712]

@mbufi which example gave you the 130 FPS on the Xavier ? , Also, how much did you get with the non asynchronous version.

I am currently running the SSD mobilenet_v2_coco on live camera feed on the Xavier (non-asynchronous) and can only reach as much as 40 FPS for some reason. I am using Jetpack 4.3 + TensorRT 6.

[marvision-ai]

@Hrayo712 I was using the aysnc version.

I believe with the non-async I was averaging ~80ish? I am using jetpack 4.2.2 with tensorRT 5.1

[Kmarconi]

Hi guys,

Just tested the ssd async version on my Xavier and i'm arround 150~170 FPS which is just amazing. But for the moment I've tested it just with my camera and there is at least 3 boxes around my person... Any advice ?

Ps: Working on Jetpack 4.3 with tensortRT 6.0

[marvision-ai]

@Kmarconi I had the same issue. Is this on live camera or on a single image?

[Kmarconi]

On live camera :)

[jkjung-avt]

Upon more thought, I just realized that 130 or 150~170 frames per second is much higher than: (a) frame rate of the camera; and (b) frame rate of the display (unless you're using a very high end industrial camera...)!!

I wasn't considering that when developing the code. So what you've seen might be the same camera image gets inferenced and drawn multiple times.

To verify whether the optimized TensorRT engine produces correct result on Jetson Xavier, please use the sync version of trt_ssd.py and look at the image frame one by one.

You might do that by, say, intentionally increasing delays between image frames. Modify this line of code. From: (1ms)

key = cv2.waitKey(1)

To: (0.1 second)

key = cv2.waitKey(100)

[Kmarconi]

It is working, yep. With the sync version and with a 25ms delay for the waitKey, I'm at 30 FPS and there is only one box for each object :) Thanks :)

[marvision-ai]

@Kmarconi Glad this was worked out :) Thanks @jkjung-avt !