Transferring the SVR function to yt

[cgyurgyik]

The intent of this project was to use it in yt. I think now is a good time to open some dialogue on what else still needs to occur for this to happen.

[matthewturk]

Hi @cgyurgyik ! @ak-2485 and I spoke today and I've spent some time working to integrate this.

Where I'm stuck is that we have a lot of building structures, which seems to take a large fraction of the time. I'm wondering if there are possibly structures that could be reused between calls to the volume walker. What we currently do is for every ray in the image, we call walkSphericalVolume, but that's not super duper efficient it turns out. For a 1024x1024 image on a 64^3 grid, the ray casting takes about a minute, and I think a lot of that time might be spent setting up lots of spherical grid objects that are identical.

Anyway, I'm attaching the first "hello world" spherical ray traced image from yt. You'll note it's just got one of the quadrants, which ... is probably something wrong on my end! :)

I'm excited about this. Really excited about it.

I've put everything in the branch spherical_vr in my repository; I actually intend to rebase this once it's done and I remove the subdirectory spherical_volume_rendering as I'd like to see if we can import your history to the repository, or link against an external one.

[cgyurgyik]

@ak-2485 Thanks for kickstarting this project again.

Hey Matt, good to talk to you again as well. Can you point me directly to this branch and/or the code that is calling walkSphericalVolume?

[matthewturk]

Yup:

https://github.com/matthewturk/yt/tree/spherical_vr

The code that calls it is in yt/utilities/lib/spherical_walk.pyx and I had to make two minor changes to the util, to add in enter_t and exit_t attributes.

[matthewturk]

In theory, we could have a setup function for each VolumeContainer, which would happen inside the ImageSampler.__call__ function.

[matthewturk]

import numpy as np
import yt
import time

values = np.ones((64,64,64), dtype="f8")

bounds = np.array([[0.0, 1.0], [0.0, np.pi], [0.0, 2*np.pi]])

ds = yt.load_uniform_grid({'density': values}, [64,64,64], bbox = bounds, geometry = "spherical")

t1 = time.time()
image = yt.volume_rendering.off_axis_projection(ds, [0.0, 0.0, 0.0], [1.0, 0.0, 0.0], 2.0, 1024, "density")
t2 = time.time()

yt.write_image(image.d, "hi.png")
print(t2-t1)

this is the example code

[matthewturk]

Uh, and actually, I take back a lot of what I said -- with regular optimizations enabled it's about 5 seconds. So, pretty pretty good.

[cgyurgyik]

Hmm yeah so I set up the cpp code so that the heavy lifting of grid-related calculations are done in SphericalVoxelGrid. However, its re-calculated each time when calling the Python version. I'm sure we can fix this with what you mentioned above, and pass in a SphericalVoxelGrid.

I know our logic for setting up bounds wasn't perfect yet, and Ariel was working on this for a bit. However, it should work fine to
traverse the entire sphere with bounds
min_bound=[0.0, 0.0, 0.0], max_bound=[max_radius, 2*pi, 2*pi] I believe. Unless, this is exactly what you did and it didn't work; then, I retract my last statement.

[matthewturk]

Awesome! Also I figured out the bounds issues - they were indeed on my end. Update soon...

…

On Tue, Aug 4, 2020, 11:59 AM Chris Gyurgyik ***@***.***> wrote: Hmm yeah so I set up the cpp code so that the heavy lifting of grid-related calculations are done in SphericalVoxelGrid. However, its re-calculated each time when calling the Python version. I'm sure we can fix this with what you mentioned above, and pass in a SphericalVoxelGrid. I know our logic for setting up bounds wasn't perfect yet, and Ariel was working on this for a bit. However, it should work fine to traverse the entire sphere with bounds min_bound=[0.0, 0.0, 0.0], max_bound=[max_radius, 2*pi, 2*pi] I believe. Unless, this is exactly what you did and it didn't work; then, I retract my last statement. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#203 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAVXOZUGW4PBWDHQ6WM6Y3R7A475ANCNFSM4NYX32IQ> .

[cgyurgyik]

I would say that we should make a full transition to one repository or another before making further changes, so we don't have duplicate copies/version control issues running around. If you can link the history, that would be great so that contributors can see our hairy progress. Not sure how easy of a task that is though.

Edit: Sounds good, looking forward to it.

[matthewturk]

OK, I'll work next on getting the history in there, although one thing I've been thinking is that maybe with the nearly 1000 commits you've got, it'd be better to either transplant it en masse (which I would invite you to do and can walk you through) or to link externally.

I figured out the issue -- was indeed on my end. I tested it with choosing a random normal vector, and here's what I get! Hooray!

[cgyurgyik]

I am ok with linking externally, or whatever you find to be the nicest (and cleanest) approach. I've never done a transfer like this, so don't have a lot of understanding on how difficult it can be.

And awesome! Will keep tabs on this. I am in the final 2 weeks of wrapping up a summer internship, so intend on contributing more once its over.

[matthewturk]

Good luck with your internship! I'm going to keep plugging away, and will try not to make too much noise here.

…

On Tue, Aug 4, 2020 at 1:36 PM Chris Gyurgyik ***@***.***> wrote: I am ok with linking externally, or whatever you find to be the nicest (and cleanest) approach. I've never done a transfer like this, so don't have a lot of understanding on how difficult it can be. And awesome! Will keep tabs on this. I am in the final 2 weeks of wrapping up a summer internship, so intend on contributing more once its over. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#203 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAVXO4NVDOSOA72R2NBSXTR7BILVANCNFSM4NYX32IQ> .