Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R-package] ensure that callbacks respect verbosity from params #5199

Merged
merged 17 commits into from
May 10, 2022

Conversation

jameslamb
Copy link
Collaborator

@jameslamb jameslamb commented May 7, 2022

Contributes to #4862.
Fixes an issue introduced by #4899 .

In the R package, lightgbm(), lgb.train(), and lgb.cv() have a keyword argument verbose. #4899 added changes to ensure that that verbosity parameters passed through params took precedence over that keyword argument.

However, evaluation and and early stopping callbacks still reference that keyword argument, which can lead to inconsistent behavior where some printed messages' verbosity can only be controlled by the keyword argument.

While working on this, I also discovered another closely-related bug...parameter verbose in calls to cb_early_stop() is being passed an integer, but that value is then checked with isTRUE()...which means that no messages are ever printed from that callback.

if (isTRUE(verbose)) {

This PR proposes changes to ensure that it's possible to silence early stopping messages and evaluation messages using params.

Notes for Reviewers

This PR reduces the number of logs printed by the R package's tests from 2095 lines to 1616 lines.

sh build-cran-package --no-build-vignettes
R CMD INSTALL --with-keep.source ./lightgbm_3.3.1.99.tar.gz
cd R-package/tests
Rscript testthat.R > ../../out.log 2>&1
echo "log lines from tests: $(cat ../../out.log | wc -l)"

@jameslamb jameslamb added the fix label May 7, 2022
@jameslamb jameslamb requested a review from StrikerRUS May 7, 2022 02:37
@jameslamb jameslamb changed the title [R-package] ensure that callbacks respect verbosity from params WIIP: [R-package] ensure that callbacks respect verbosity from params May 7, 2022
@jameslamb jameslamb changed the title WIIP: [R-package] ensure that callbacks respect verbosity from params WIP: [R-package] ensure that callbacks respect verbosity from params May 7, 2022
@jameslamb jameslamb marked this pull request as ready for review May 8, 2022 21:29
@jameslamb jameslamb changed the title WIP: [R-package] ensure that callbacks respect verbosity from params [R-package] ensure that callbacks respect verbosity from params May 8, 2022
Copy link
Collaborator

@jmoralez jmoralez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very thorough tests!

Copy link
Collaborator

@StrikerRUS StrikerRUS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

Comment on lines 196 to 199
# Set validation as oneself
if (verbose > 0L) {
if (params[["verbosity"]] > 0L) {
train_args[["valids"]][["train"]] <- dtrain
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite strange behavior, to be honest...
Maybe remove it in v4.0.0?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I totally agree! I definitely think we should consider removing this prior to v4.0.0.

This has been in {lightgbm} since the very first version of the R package, in #168.

Wrote up a separate issue for discussion of that: #5203.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for doing this!

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants