Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding source detection step #608

Merged
merged 21 commits into from
Mar 24, 2023

Conversation

cshanahan1
Copy link
Collaborator

@cshanahan1 cshanahan1 commented Dec 6, 2022

Resolves RCAL-423

Adds the source catalog step, which uses DAOStarFinder to detect point sources and writes out and returns a catalog.

Checklist

  • added entry in CHANGES.rst under the corresponding subsection
  • updated relevant tests
  • updated relevant documentation
  • updated relevant milestone(s)
  • added relevant label(s)

@nden nden added this to the 23Q2_B9 milestone Dec 17, 2022
@github-actions github-actions bot added documentation Improvements or additions to documentation ramp_fitting labels Jan 26, 2023
@nden
Copy link
Collaborator

nden commented Jan 26, 2023

@cshanahan1 Please fix style changes and rebase

@github-actions github-actions bot added the dependencies Pull requests that update a dependency file label Feb 7, 2023
@schlafly
Copy link
Collaborator

schlafly commented Feb 8, 2023

This looks reasonable to me. It looks like the style failure has to do with some kind of configuration error with the new pre-commit step? And then we need to add photutils as a dependency so that the tests can run?

@ddavis-stsci ddavis-stsci modified the milestones: 23Q2_B9, 23Q3_B10 Feb 27, 2023
@github-actions github-actions bot added testing and removed dependencies Pull requests that update a dependency file labels Mar 9, 2023
@github-actions github-actions bot added dependencies Pull requests that update a dependency file and removed ramp_fitting labels Mar 15, 2023
@codecov
Copy link

codecov bot commented Mar 16, 2023

Codecov Report

Patch coverage: 72.72% and project coverage change: +0.17 🎉

Comparison is base (6cac937) 63.13% compared to head (66c508c) 63.31%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #608      +/-   ##
==========================================
+ Coverage   63.13%   63.31%   +0.17%     
==========================================
  Files          76       78       +2     
  Lines        4036     4113      +77     
==========================================
+ Hits         2548     2604      +56     
- Misses       1488     1509      +21     
Flag Coverage Δ *Carryforward flag
nightly 63.07% <ø> (ø) Carriedforward from 3b312cb

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files Coverage Δ
romancal/lib/suffix.py 94.28% <ø> (ø)
romancal/pipeline/exposure_pipeline.py 88.88% <50.00%> (-1.12%) ⬇️
romancal/source_detection/source_detection_step.py 72.60% <72.60%> (ø)
romancal/source_detection/__init__.py 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@cshanahan1 cshanahan1 force-pushed the source_detection branch 4 times, most recently from d362010 to 7e2e34a Compare March 17, 2023 18:50
Copy link
Collaborator

@mairanteodoro mairanteodoro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

romancal/source_detection/source_detection_step.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@schlafly schlafly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I added some comments in-line that may resolve the half-pixel offsets you're seeing. Most important outstanding questions for me are:

  • Do you still think there are especially fragile bits of source detection that we should be tracking? I think I'd probably record them with comments.
  • I had the impression in our testing that we found that subtracting the background and then photometering the sky-subtracted image was better performing than including the background in the threshold. Has your view there developed?

docs/roman/source_detection/arguments.rst Outdated Show resolved Hide resolved
docs/roman/source_detection/arguments.rst Outdated Show resolved Hide resolved
docs/roman/source_detection/description.rst Outdated Show resolved Hide resolved
romancal/source_detection/source_detection_step.py Outdated Show resolved Hide resolved
@cshanahan1 cshanahan1 force-pushed the source_detection branch 2 times, most recently from 5fe024b to 9200cda Compare March 23, 2023 16:31
@cshanahan1
Copy link
Collaborator Author

is this ready to merge?

@mairanteodoro
Copy link
Collaborator

mairanteodoro commented Mar 24, 2023

Should we merge RAD PR#215 and roman_datamodels PR#132 first?

@cshanahan1
Copy link
Collaborator Author

The only checks here failing are codecov. I did write tests that should cover every line, but they are not enabled under CI because they require a connection to roman CRDS. Is there a way to run the step and not require this connection so the tests can run? this step doesnt require a reference file.

@mairanteodoro
Copy link
Collaborator

The only checks here failing are codecov. I did write tests that should cover every line, but they are not enabled under CI because they require a connection to roman CRDS. Is there a way to run the step and not require this connection so the tests can run? this step doesnt require a reference file.

I think that if the test doesn't need a reference file then it should be safe to remove theskipif decorator.

@cshanahan1
Copy link
Collaborator Author

The only checks here failing are codecov. I did write tests that should cover every line, but they are not enabled under CI because they require a connection to roman CRDS. Is there a way to run the step and not require this connection so the tests can run? this step doesnt require a reference file.

I think that if the test doesn't need a reference file then it should be safe to remove theskipif decorator.

I tried skipping the decorator but it still tries to make the connection

@mairanteodoro
Copy link
Collaborator

If the tests pass, I'm good with merging this PR.

@cshanahan1
Copy link
Collaborator Author

i'm going to merge this once checks pass

@cshanahan1 cshanahan1 merged commit dec39b6 into spacetelescope:main Mar 24, 2023
mairanteodoro pushed a commit to mairanteodoro/romancal that referenced this pull request Mar 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation pipeline source_detection testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants