Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document decision tree steps in report and remove log_extra_report #1043

Merged
merged 37 commits into from
Feb 23, 2024

Conversation

handwerkerd
Copy link
Member

@handwerkerd handwerkerd commented Feb 21, 2024

This is finishing up #959 on a branch that I can more easily push onto.

Closes None.

Changes proposed in this pull request:

  • Decision tree json files have more detailed _comment fields explaining each step of the decision tree
  • The decision tree report line now references 10.6084/m9.figshare.25251433 The updated data are at: https://figshare.com/account/articles/25251433 (not public yet) which will be a way to archive specific decision trees that people use in publications.
    • This is a new entry in references.bib along with olafsson2015enhanced One issue of note is that I was required to include an author in references.bib for the figshare DOI to be rendered properly. Unless there is a way to avoid this, I put the author in as tedana community
  • Since decision tree steps will no longer write out node-specific text into the report, log_extra_report was removed from the code, including tests.
    • This also involved updating the data used for testing ica_reclassify so that desc-ICA_decision_tree.json no longer includes the log_extra_report field. I'm fairly sure this won't mess up tests on other branches.
    • Documentation was also updated to reflect that removal of log_extra_report

There is one remaining testing failure that I might need help figuring out. The five-echo integration test is the only one that uses the minimal decision tree. That report field includes the line \\citep{kundu2013integrated, dupre2021te} https://github.com/handwerkerd/tedana/blob/09b9e2ef7f79262fad9ab7a32207012b51304ad6/tedana/resources/decision_trees/minimal.json#L4 For some reason the test is failing due to the comma separated citations. I think this isn't new to the code so I'm not sure why it's failing now. Any ideas? Solved. There can't be a space after the comma between the two reference keys.

@handwerkerd handwerkerd added the refactoring issues proposing/requesting changes to the code which do not impact behavior label Feb 21, 2024
Copy link

codecov bot commented Feb 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.34%. Comparing base (d0298f2) to head (e22c963).
Report is 42 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1043      +/-   ##
==========================================
- Coverage   89.55%   89.34%   -0.22%     
==========================================
  Files          26       26              
  Lines        3466     3444      -22     
  Branches      634      622      -12     
==========================================
- Hits         3104     3077      -27     
- Misses        210      213       +3     
- Partials      152      154       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@handwerkerd
Copy link
Member Author

I think this is ready to merge. In addition to having 2 people review the code, anyone should be able to look at this link for figshare: https://figshare.com/s/8e62b5555a9f50a4ad43 Once I hit "publish" on figshare I won't be able to change anything in that version so I'd appreciate multiple eyes (@tsalo @eurunuela @dowdlelt @n-reddy @goodalse2019 ) to make sure it is what we want this to look like. The goal is, when tedana is run, it will include a DOI that links to that figshare which will have the versions of the decision trees that were run by tedana. Whenever we update the decision trees, we can publish a new version and then anyone who processes data with the new version will be directed to the newer link.

eurunuela
eurunuela previously approved these changes Feb 22, 2024
Copy link
Collaborator

@eurunuela eurunuela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with the changes to the code.

Edit: I'm happy with FigShare too.

Copy link
Member

@tsalo tsalo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few small recommended changes, but other than that this looks good.

I don't think log_extra_info is really necessary, TBH. It's mostly redundant with _comment, and for the most part the information that is included in those fields doesn't seem like information that needs to be written out to the terminal.

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>
eurunuela
eurunuela previously approved these changes Feb 22, 2024
tsalo
tsalo previously approved these changes Feb 22, 2024
Copy link
Member

@tsalo tsalo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@handwerkerd handwerkerd dismissed stale reviews from tsalo and eurunuela via e22c963 February 22, 2024 16:33
@handwerkerd
Copy link
Member Author

handwerkerd commented Feb 22, 2024

We're not using log_extra_info much right now, but I'm inclined to keep it because it might help highlight steps in the log that aren't as clear with the automatic messages.

One thing that just occurred to me is that, because I removed log_extra_report, if someone runs a tree that uses that field it will crash, which would make this PR a breaking change that might annoy some users. I decided to add a warning so that it won't crash. Sorry for asking for another (hopefully) quick review. The new changes are in component_selector.py and test_component_selector.py

@dowdlelt
Copy link
Collaborator

I looked through the png files and didn't seen any typos or anything. My lack of familiarity with the decision trees themselves means I can't check much beyond that, but they looked complete. Confusing of course, but thats the nature of the software - and it means users can have more than. what they have now.

@handwerkerd handwerkerd merged commit 1f1845b into main Feb 23, 2024
16 checks passed
@handwerkerd handwerkerd deleted the doc-tree branch February 23, 2024 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactoring issues proposing/requesting changes to the code which do not impact behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants