Alg_roi.jpg store into memory directly as JPG

[Slider0007]

The Feature

Store alg_roi.jpg into memory directly as JPG

• Reduce memory usage
• Faster refresh

[caco3]

Maybe this could be done together with #1187

[caco3]

As far as i understand, the framebuffers are directly JPG.
At least for the demo mode, i simply replace the framebuffer with a read JPG and it just works

[Slider0007]

These are my ideas:
Right now, the ROI overlays are drawn over a CImageBasis object every time the alg_roi is requested (Processed in ClassFlowControll::GetJPGStream routine). My idea is to use the already existing ImageTMP CImageBasis object which is needed for alignment anyway and draw ROIs in this object after alignment is done. The result can be stored as JPG in RAM and recalled when it's needed without reprocessing it again.

Advantages:

• less memory due to the fact no additional CImage Object is needed for ROI (save 921kB RAM)
• faster refresh
• ROI only visible when alignment is done
• In early phase till alignment is done, raw image can be shown

I provide a draft the next days, then you can test and have a look to the proposal.

#1187: This means some text needs to be included in the image, right? Do you have an idea / library how to include this?

[caco3]

921kB RAM

You are a genius 😺

This means some text needs to be included in the image, right?

Hmm, I see your problem, maybe we simply could draw an icon, eg. for alignment we could show something like:

[Slider0007]

Hmm, I see your problem, maybe we simply could draw an icon, eg. for alignment we could show something like:

The image manipulation is done by setting single pixels in an image array (CImageBasis::setPixelColor). Whatever element you'd like to include you have to program it pixel by pixel. As long there is no library, this could be a lot of work. As far as I can see right now there are only basic elements like line, rectangle, circle, elipse. A text library would be great...

[caco3]

but a text library again adds RAM usage, drawing a simple icon might be more efficient

[caco3]

so if i understand it right, before

AI-on-the-edge-device/code/components/jomjol_flowcontroll/ClassFlowControll.cpp

Line 649 in fca37ee

if (flowdigit) flowdigit->DrawROI(_imgzw);

we should check in which step we are and only draw the ROIs if we completed the alignment step.

Looking at

AI-on-the-edge-device/code/components/jomjol_flowcontroll/ClassFlowControll.cpp

Line 76 in fca37ee

std::string ClassFlowControll::TranslateAktstatus(std::string _input)

this should not be too hard.
And maybe we can dram a simple camera while in the Take Picture step (althrough I am not sure what currently gets drawn in that step).

Looking on my MQTT Log, Take Image takes around 8s, so there is a chance to catch this step:

[caco3]

If you want, I can create icons for those 2 cases

[Slider0007]

In my opinion we have the status text which tells what happens right now. Maybe it's helpful to have status text more prominent if flow is actually processing...

Showing the ROI overlay only after the alignment completed is for sure beneficial to avoid some confusion...

[Slider0007]

This would be the view before alignment is completed:

[caco3]

Yes, an overlay on the UI would be ok too, but is tricky to implement.

The Web UI only works with polling, which means that the status information might not get loaded at the same time as the image.
So those changes most likely are not fully in sync, causing confusion.

Also one could pull the image separately and eg. show it in Homeassistant. Then we would miss the icons

[Slider0007]

Yes, an overlay on the UI would be ok too, but is tricky to implement.

The Web UI only works with polling, which means that the status information might not get loaded at the same time as the image. So those changes most likely are not fully in sync, causing confusion.

The image on the webinterface is also only updated with polling (e.g. page refresh). With the refresh of the page or automatic refresh after the specified time the status text is going to be updated, as well (like it's done already)
Or do you mean the sync while the loading / refesh is already initiated because of loading delay?

[jomjol]

Writing text to the image: I use std_image.h as a simple library. This does not provide writing text to an image, only pixel manipulation.

[caco3]

doing it directly on the image would save us the hassle with the UI only partially working//updating!

I would be happy to provide some drawing instructions

[Slider0007]

For sure you can provide some drawings if you want :-)

[caco3]

@Slider0007 During the step Take Image the image is just black, right?
If so, we could just serve an image from the SD Card instead. We would save to have to draw an icon in that case.
What do you think?

[Slider0007]

@caco3: Actually, in the step "Take Image" it starts with a black image and then you see the image building up line by line, but this takes only a few secs before it switches to alignment step. Therefore this "image capture process" could be skipped for sure. I was also thinking about this during my coding.

Providing an alternative image during "Take Image" step would be an option, also to avoid the black screen at the beginning of the process.

What is also missing is the visualization when flow is not started yet. Also for this we could show an indication image which is loaded from SD.

Possible steps:

1. Device / webserver started, but flow not started (yet) --> Image with indication "flow not started"
2. Flow started and step "Take image" --> Image with indication of taking image
3. Flow started and step "Alignemnt" --> serve alg / alg_roi (like already coded + ?alignment icon?)
4. and the follwoing steps --> serve alg / alg_roi (like already coded)

What do you think?

[caco3]

I like your proposal!

Just seeing a lack picture can really be confusing, there where already a few questions because of this.
And another special image in case the flow is not started yet would also be helpful.