wip Feat/norm16 textures

[sedghi]

This relies on this cornerstonejs/cornerstoneWADOImageLoader#500

Volume Viewport

• I made the necessary changes, but seems like the part where we update texture in fillImage3D has some problem, when I run volumeAPI example, the webGL context get lost, and I get a flash of empty screen

https://github.com/cornerstonejs/cornerstone3D-beta/blob/feat/norm16-textures/packages/core/src/RenderingEngine/vtkClasses/vtkStreamingOpenGLTexture.js#L103

Stack Viewport

• stack viewport not working need to be fixed. I guess we can just use Float32 for now? as before. Later we optimize this? Oh yes, I guess I didn't add vtk.js support for imageMapper for norm16, so we don't have option, we should just use it as before

CPU

• Needs fixing I guess since I commented out bunch of things. I don't think we need to do anything here, just uncomment what I commented out

CC @swederik @Ouwen It is very interesting, sometimes when I loose the GL context, the extension becomes unavailable (webgl report v2) I need to restart chrome for it to appear again!)

[netlify]

✅ Deploy Preview for cornerstone-3d-docs ready!

Name	Link
🔨 Latest commit	dc2366e
🔍 Latest deploy log	https://app.netlify.com/sites/cornerstone-3d-docs/deploys/63a376a92ac9280009635559
😎 Deploy Preview	https://deploy-preview-336--cornerstone-3d-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[sedghi]

This needs to get expanded to consider scaling parameters to decide on the final data type (if scalings are float, then the type passed to wado as option should be float)

[Ouwen]

I am assuming this is usually used for PET overlays. Would it be worth it to account for preferSizeOverAccuracy to perform a scale + cast

[Ouwen]

CC @swederik @Ouwen It is very interesting, sometimes when I loose the GL context, the extension becomes unavailable > (webgl report v2) I need to restart chrome for it to appear again!)

That is weird behavior... is the GL context lost just during debugging or will it randomly occur. I can try it on my local too

[Ouwen]

@sedghi the divergence of stack and volume might have some consequence for the cache since the stored values are going to be uint16 / int16 in the cache.

Is the plan just to cast them to float32 inside the stackViewport on vtkImageData creation? Then the CPU cache can be small.

StackViewport doesn't seem to use too much GPU memory anyway so I don't think it is a big deal

[sedghi]

@Ouwen That is an interesting point, maybe then I just add the norm 16 to image mapper as well.

I'm having some trouble with the rendering volume right now on Mac and Chrome. If you have time and look for some headache, can you check this branch (with the changes in wado, PR as well there), and see if volumeAPI example work for you?

I'm looking inside Chromium forums to see if it has been reported there too.

[Ouwen]

ha nice abstract art feature, is this also what you are seeing so far?

[Ouwen]

@sedghi did a rebase of this on top of
https://github.com/gradienthealth/cornerstone3D-beta/tree/gradienthealth/no_shared_array_buffer

I made some changes to the stack viewport so it should also properly render int16 and uint16 data types. Apologies on the premature commit squashing for now...
https://github.com/gradienthealth/cornerstone3D-beta/tree/gradienthealth/norm16-textures

One of the issues I found was that uint16 support requires some more logic for prescaling.
I think that we should perform some casting logic here
https://github.com/cornerstonejs/cornerstoneWADOImageLoader/blob/9c14da52e0ddd8224d2e0973105895d0bc6b6b19/src/shared/scaling/scaleArray.js#L1-L21

Thoughts on the following:
if a targetbuffer type is present it means we should always return that targetbuffer type. Thus, if the target is uint16 and scaling values are an int we may need to cast the pixeldata to int16, scale, then recast to uint16.

if a targetbuffer is not present we are using the native decoded pixelData (float32, uint8, uint16, int16, int8 not supported yet but maybe later). We try to preserve dynamic range. So if rescale slope and rescaleIntercept are both positive and native type is uint16, we can keep uint16 type. However, if rescale slop and rescaleIntercept are possibly negative we can cast to int16. Alternatively, we can always cast to float32 and recast to the min pixel type to preserve dynamic range (keep track of a min/max)

[sedghi]

@Ouwen great observations.
Yes, I was thinking about the same thing couple of days ago
IMO, we shouldn't force the targetBuffer to be float32, we should just let the wado detect and choose the correct type. Why do you think there is a use case for specifying the targetBuffer type? it should just be the array buffer. no?

[Ouwen]

I think the only reason target buffer is needed is because for shared array buffers we need to allocate prior to decoding for progressive loading.

So to allocate the best type, we'd need to look at scaling, minvalue, maxvalue, and some other dicom metadata tags (signed, bitsAllocated, photometric interpertation)

[Ouwen]

this can be closed with #420