Add flux normaliser processor

[hrobarts]

Description

New FluxNormaliser processor to normalise data

• to a float value
• to an array of floats with size matching the
• to the mean value calculated from a specified region of interest in the data itself

Example Usage

Changes

Testing you performed

Please add any demo scripts to https://github.com/TomographicImaging/CIL-Demos/tree/main/misc
Tests in test_DataProcessor.py TestFluxNormaliser class

Related issues/links

#1877

Checklist

• I have performed a self-review of my code
• I have added docstrings in line with the guidance in the developer guide
• I have updated the relevant documentation
• I have implemented unit tests that cover any new or modified functionality
• CHANGELOG.md has been updated with any functionality change
• Request review from all relevant developers
• Change pull request label to 'Waiting for review'

Contribution Notes

Please read and adhere to the developer guide and local patterns and conventions.

• The content of this Pull Request (the Contribution) is intentionally submitted for inclusion in CIL (the Work) under the terms and conditions of the Apache-2.0 License
• I confirm that the contribution does not violate any intellectual property rights of third parties

[hrobarts]

Some examples:
Normalise all data with a scalar value

processor = FluxNormaliser(flux=10)
processor.set_input(data)
data_norm = processor.get_output()
show2D([data, data_norm], ['Data','Normalised data'])

Normalise data with a scalar value per projection

processor = FluxNormaliser(flux=np.arange(1,2,(2-1)/(data.get_dimension_size('angle'))))
processor.set_input(data)
data_norm = processor.get_output()
show2D([data, data_norm], slice_list=('vertical',80))

Normalise data using a region in the background of each projection.

roi = {'vertical':(0,14), 'horizontal':(0,14)}
processor = FluxNormaliser(roi=roi)
processor.set_input(data)
processor.show_roi()

The default output of show_roi() shows the roi plotted on the dataset for the first and last projection and the projection which has the maximum variation from the mean intensity (Z-score) in the roi, as well as a plot of the mean intensity per projection.

In this case we can see that although the roi is empty for most projections, there are some cases where there is material in the roi like index 176.
We can also use processor.show_roi(angle_index=250) with a specific angle_index if we want to check a different projection which looks like an outlier

In this case, if we change the roi we can make sure it's empty for all projections

roi = {'vertical':(0,10), 'horizontal':(0,10)}
processor = FluxNormaliser(roi=roi)
processor.set_input(data)
processor.show_roi()

data_norm = processor.get_output()

In this example with synchrotron data, the background has a gradient so the Z-score might not be the best metric to use.

roi = {'vertical':(10,135), 'horizontal':(125,150)}
processor = FluxNormaliser(roi=roi)
processor.set_input(data)
processor.show_roi()

roi = {'vertical':(10,135), 'horizontal':(135,150)}
processor = FluxNormaliser(roi=roi)
processor.set_input(data)
processor.show_roi()

data_norm = processor.get_output()
show2D([data, data_norm], slice_list=('vertical',80))

The gradient is opposite if we use the roi on the other side

roi = {'vertical':(10,135), 'horizontal':(10,32)}
processor = FluxNormaliser(roi=roi)
processor.set_input(data)
processor.show_roi()

data_norm = processor.get_output()
show2D([data, data_norm], slice_list=('vertical',80))

[hrobarts]

Updated to plot the mean, minimum and maximum in the roi

[hrobarts]

Updated so preview_configuration() still works on data with one angle

Also works on 2D data where the roi is specified with one of horizontal or vertical. Plot the roi on the sinogram

Also works if the dataorder is different

Also works if the data has multiple channels, the flux to normalise to is the mean in the roi averaged over the channels. preview_configuration() shows the central channel by default

or the channel can be specified processor.preview_configuration(channel=0)

[hrobarts]

@gfardell we discussed whether the preview_configuration should just be an option in logging. At the moment I've left it as a function the users can access - what do you think?

[lauramurgatroyd]

This is looking great and super useful! Excellent documentation in this PR to go along with it as well. as I have said in my comments, we need to make sure it lives somewhere useful!

I added some comments, mostly to improve the unit tests a bit and add some docs

[hrobarts]

Hi @lauramurgatroyd thank you for your review! I have made lots of updates

• More efficiently access data in process (following conversation with @gfardell )
• Error if horizontal and vertical are not in the last two dimensions of the data
• Change norm_value to target and allow it to be passed as a number or string mean or first which gets the target from the mean or first value of the flux array (e.g. this could be the mean value in the roi across all projections)
• Moved _calculate_flux and _calculate_target to separate functions out of check_input
• New tests to test this functionality

I have added the notebook where I created the examples above to CIL-Demos/misc in this PR TomographicImaging/CIL-Demos#187