Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More metric summarizer #322

Merged
merged 45 commits into from
Oct 20, 2022
Merged

More metric summarizer #322

merged 45 commits into from
Oct 20, 2022

Conversation

EmilHvitfeldt
Copy link
Member

@EmilHvitfeldt EmilHvitfeldt commented Sep 27, 2022

This PR adds a refactoring of metric_summarizer() by splitting it into 3 new functions metric_summarizer_class(), metric_summarizer_numeric(), and metric_summarizer_prob(). (metric_summarizer() is being soft deprecated)

This will lessen the burden of modifying any one of the types of metrics as metric_summarizer() was forced to cover all cases. This PR is one step towards adding survival metrics.

metric_summarizer_class() and metric_summarizer_prob() are carbon copies of metric_summarizer(), where on the other hand , metric_summarizer_numeric() doesn't have an estimator or event_level arguments as they aren't used for numeric metrics.

I decided to document them separately, as I found it less confusing to have for numeric metrics this is NULL, for class metrics ....

@EmilHvitfeldt
Copy link
Member Author

I have not contacted revdep authors yet. Will do that before merging

Copy link
Member

@DavisVaughan DavisVaughan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I've given a first round of review. I am fully expecting this to be a rather large PR with a few iterations. Feel free to ping me with questions, and when you finish this round

R/template.R Outdated
Comment on lines 175 to 178
truth = !! truth,
estimate = !! estimate,
na_rm = na_rm,
!!! spliceable_case_weights(case_weights),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've got some ideas to make truth, estimate, case_weights, and ... use tidyselect rather than allowing arbitrary expressions through here (which in retrospect is a bit horrifying). We can talk about it after this first round of review, but I think it will simplify these functions a bit further.

If I'm right, we will be able to do something like .data[[truth]] rather than !! which would be nice

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that will also automatically support this special case so we would be able to remove

  # Explicit handling of length 1 character vectors as column names
  nms <- colnames(data)
  truth <- handle_chr_names(truth, nms)
  estimate <- handle_chr_names(estimate, nms)

That would work automatically because tidyselection allows strings as well as bare expressions

R/template.R Outdated
Comment on lines 175 to 178
truth = !! truth,
estimate = !! estimate,
na_rm = na_rm,
!!! spliceable_case_weights(case_weights),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hoping to remove all of the spliceable_case_weights() style helpers too, which we can also talk about

- metric_nm -> name
- metric_fn -> fn
- metric_fn_options -> fn_options

also changed the templateVar to use this convention
Copy link
Member

@DavisVaughan DavisVaughan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is looking good! One more round of review. In the meantime I'll also think about the tidyselect bits more and send you a message about it.

@EmilHvitfeldt EmilHvitfeldt merged commit cf8bdab into main Oct 20, 2022
@EmilHvitfeldt EmilHvitfeldt deleted the more-metric_summarizer branch October 20, 2022 19:56
@github-actions
Copy link

github-actions bot commented Nov 4, 2022

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Nov 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants