Add binned_histogram

[Lazersmoke]

This isn't finished/polished yet (opening this PR to track progress).

This is the alternative interface to intensities which is designed to more closely align with what Mantid/Horace output. In particular, the new type BinningParameters is supposed to be isomorphic to the data Mantid uses to describe an "MDHistoWorkspace".

Example usage:

# Histogram a 1D cut with from [0,1,0] to [1/2,0,0] (with transverse width 0.3)
params = Sunny.one_dimensional_cut_binning_parameters(sf,[0,1,0],[1/2,0,0],30,0.3)
display(params)

# sunHist: Histogramed intensity data
# sunCounts: Number of scattering vectors per bin (NOT uniform)
sunHist,sunCounts = Sunny.binned_histogram(sf,params,Sunny.DipoleFactor(sf),kT,formfactors)

# Axes labelled by bin centers
mx = Sunny.axes_bincenters(params)

fig3 = Plots.heatmap(mx[1],mx[4],sunCounts[:,1,1,:]')
logIntensity = log10.(sunHist[:,1,1,:]' ./ sunCounts[:,1,1,:]')
fig4 = Plots.heatmap(mx[1],mx[4],logIntensity;ylabel="E (meV)",color=cgrad(:viridis,scale = :lin,rev = false),xticks = ([mx[1][1],mx[1][end]], ["[0,1,0]","[1/2,0,0]"]))

display(Plots.plot(fig3,fig4;layout = @layout([a{0.2h};b])))

The resulting intensity histogram looks something like this:

[Lazersmoke]

TODO's on this include:

• Needs a separate function to treat powder averaging, since BinningParameters as written only makes sense for linear coordinates, and spheres are nonlinear
• Automatic conversion of Mantid/Horace output files (in HDF format) into BinningParameters objects. The manual conversion is not particularly difficult, but it is annoying to get right.
• Make it work with SpinWaveTheory. There are two possible ways to implement this, depending on whether we trust the spin wave theory calculation at q vectors which are not compatible with the periodic boundary conditions.

[kbarros]

This sounds relevant to @ddahlbom

[ddahlbom]

This looks great. Thanks for the PR.

I do think DataRetrieval.jl is the right place for it. We may expand this into a folder with more fine-grained divisions in the future.

I was a bit skeptical on the call as to whether all the components should be stored in a single covector (as opposed to explicitly treating q and omega separately, as in the rest of the code), but it does result in cleaner code even if there are no relativistic transformations.

I guess you'll have to write a different binned_histogram(sw:SpinWaveTheory, ...). Out of curiosity, what will be the approach there? Will you perform some kind of sampling/integration over the bins?

I put some (trivial) comments in the code.

[ddahlbom]

Perhaps integrate_parameters!, since "parameters" is written out in full elsewhere. Or maybe integrate_axes! is more precise. That it is operating on binning parameters is implied by the argument type.

[ddahlbom]

This is a neat pattern. I hadn't seen it before.

[Lazersmoke]

Yep! One slight issue is that code like params.numbins[4] = 1 doesn't work how you would expect (it's a no-op). I don't know if that's a deal breaker for this interface, but it's papered over by integrate_params! currently.

[ddahlbom]

I know these function names are getting long, but maybe write out resolution in full.

[Lazersmoke]

Additional TODO on this, per discussion: need to incorporate broadening into the inner histogram loop, same as LSWT code.

[Lazersmoke]

@ddahlbom Currently, the binned intensities functions (linear and spherical) do not handle the form factors, contractors, and possibly kT correctly, since they bypass the logic in intensities. What do you + @kbarros recommend to do about the interface for this? I may just finish implementing the binning internals and then turn it over to y'all for making the final user interface arrangements for these parameters (e.g. following the google doc)

[ddahlbom]

@Lazersmoke Sure, that sounds great. Just let us know when when you're done making the final touches. I'd be happy to incorporate the rest of the intensities logic afterwards.

[Lazersmoke]

OK, the final touches are done :)

I've added examples of binned version to the FeI2 and powder averaging tutorials. Some of the plotting is a bit rough around the edges.

There's a remaining question about the exact behavior of the "partial bins" at the far edge of the histogram, sometimes the behavior is possibly not quite correct, but the issue is that it's a bit under-specified what should be done in that case. Mantid has a bunch of options on like every function call that specify what to do with them, seems like. It shouldn't be a big issue for most use cases.

Remaining items for @ddahlbom and co:

• Documentation is in comment format right now, not sure on best practices for that
• The contractor/ffdata/kT interface needs to be bolted on the front of everything in an appropriate way
• The interface for the integrated_kernel could be improved, as per discussion in slack
• There is some code duplication between the powder-averaged and linear histograms; the logic is somewhat subtle surrounding the energy binning so one has to be careful, but they could possibly be combined by passing a get_coordinates function which generalizes both covectors * and norm(q). Relatedly: The covectors are in Q space by default, but Mantid is in k space (absolute units). It may be advantageous to be in k space by default for that reason.

[ddahlbom]

Thanks @Lazersmoke ! This is a great addition. I'll take a closer look at the code soon. If there are any changes that seem like they should be made before merging, I will comment here.

(I suppose we may want to merge sooner rather than later to keep things in sync. I can complete the remaining tasks after this is integrated.)