Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add calibration calculators #2609

Open
wants to merge 111 commits into
base: main
Choose a base branch
from
Open

Add calibration calculators #2609

wants to merge 111 commits into from

Conversation

TjarkMiener
Copy link
Member

@TjarkMiener TjarkMiener commented Aug 26, 2024

This PR contains the StatisticsCalculator, which aggregates statistics, detects outliers, handles faulty data chunks. The StatisticsCalculator holds two functions to conduct two different passes over the data with and without overlapping chunks. The first pass is conducted with non-overlapping, while overlapping chunks can be set by the chunk_shift parameter in the second pass. The second pass over the data is only conducted in regions of trouble with a high percentage of faulty pixels exceeding the threshold faulty_pixels_threshold.

related to issue #2542

Tjark Miener and others added 30 commits April 29, 2024 08:51
added PlainExtractor based on numpy and scipy functions
restructured the stats containers
Remove StarVarianceExtractor since is functionality is featured in the existing Extractors
allow overlapping extraction sequences
renaming to chunk(s) and chunk_size and _shift

added test for chunk_shift and boundary case
)
# Check if at least one chunk is faulty
if np.all(valid_chunks):
raise ValueError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really an error? Or should it just return as a no-op?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check is needed because if all chunks are valid, an empty list would be vstack later on, which results an error without a proper message. I would prefer to break here and put a check for the validity of chunks outside this function, to avoid having multiple returns in one function. I find it really hard to follow the logic of a function with multiple returns (maybe just a personal taste). If you insists I can modify it to return a no-op. However, if everything went well in the first_pass, the second_pass should not be called in my opinion.

@maxnoe
Copy link
Member

maxnoe commented Oct 11, 2024

After this, we are still missing a Component and/or Tool to perform the actual computations on input data containing the calibration events, correct?

And that will finally replace the two classes in ctapipe.calib.camera.{flatfield,pedestal}?

@TjarkMiener
Copy link
Member Author

After this, we are still missing a Component and/or Tool to perform the actual computations on input data containing the calibration events, correct?

Correct, we have prototype tools ready in calibpipe-MRs for the stats calculation and the ffactor method.

And that will finally replace the two classes in ctapipe.calib.camera.{flatfield,pedestal}?

Yes, we would have to make sure that the interface to camcalib coefficients is in place before removing them though.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link

Passed

Analysis Details

0 Issues

  • Bug 0 Bugs
  • Vulnerability 0 Vulnerabilities
  • Code Smell 0 Code Smells

Coverage and Duplications

  • Coverage 97.40% Coverage (94.30% Estimated after merge)
  • Duplications 0.00% Duplicated Code (0.70% Estimated after merge)

Project ID: cta-observatory_ctapipe_AY52EYhuvuGcMFidNyUs

View in SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants