[wpiutil] Add a RawFrame JNI overload for byte[]

[spacey-sooty]

Allows avoiding two copies in #7176

cc @mcm001

[spacey-sooty]

/format

[spacey-sooty]

If desired an overload which just takes the Mat.dataAddr return shouldn't be too hard here but I believe this should fix the issue with extra copies as it keeps the byte[] overload #7176 adds but removes the copies incurred.

[mcm001]

Yeah this gets rid of one copy, but for use with opencv mats we need an overload that takes (long pData, int width/height/stride/format). Not sure if there's any other implications for making a RawFrame which doesn't own the underlying array.

[spacey-sooty]

Yeah this gets rid of one copy, but for use with opencv mats we need an overload that takes (long pData, int width/height/stride/format). Not sure if there's any other implications for making a RawFrame which doesn't own the underlying array.

Hmm Peter raised concerns about that, how bad would the performance implications of the copy be in PhotonVision usecase? Is there any way to use RawFrame over Mat there?

[mcm001]

I don't think that the Mat constructor that takes a pointer to C++ backed data is accessible. Also cscore's USBCamera gives us a Mat right? Not a RawFrame? I will need to convert one to the other at some point, and it's not workable to incur an extra copy at any step (it's on the order of ms on embedded devices)

[mcm001]

What's the concern with making RawFrame not own the underlying data? Opencv's mats let you do this.

[spacey-sooty]

I don't think that the Mat constructor that takes a pointer to C++ backed data is accessible. Also cscore's USBCamera gives us a Mat right? Not a RawFrame? I will need to convert one to the other at some point, and it's not workable to incur an extra copy at any step (it's on the order of ms on embedded devices)

We could change USBCamera to give a raw frame

[mcm001]

If we do that, someone should provide a way to get that data into a opencv Mat without extra copies happening. Thats possible in photon via our new JNI capabilities, I guess, if we wanna commit to nuking opencv entirely.

[Gold856]

USBCamera doesn't give a Mat by itself, it's instead added as a source to a CvSink, and the Mat is pulled from that. We could instead use a RawSink, which will allow us to use a RawFrame instead of a Mat.

As for the dangers of taking the pointer stored in the Mat and using it in RawFrame, Mat uses finalize to clean up, so the data should last until the Mat gets GCed. For what we're doing, I think this is long enough?

[mcm001]

It looks like the Java matrix wrapping the RawMat being released isn't a concern? See the release javadoc claim: https://docs.opencv.org/3.4/d3/d63/classcv_1_1Mat.html#ae48d4913285518e2c21a3457017e716e

If the matrix header points to an external data set (see Mat::Mat ), the reference counter is NULL, and the method has no effect in this case.

Also in Mat::Mat:

The external data is not automatically deallocated, so you should take care of it.