Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prettier warning output #18288

Merged
merged 3 commits into from
Sep 19, 2023
Merged

Prettier warning output #18288

merged 3 commits into from
Sep 19, 2023

Conversation

awaelchli
Copy link
Contributor

@awaelchli awaelchli commented Aug 11, 2023

What does this PR do?

This PR proposes a prettier warning printing. Currently, the user sees a warning formatted like this, with all the text crammed in a single line and a strange rank_zero_warn( on a new line:

/Users/adrian/repositories/lightning/src/lightning/pytorch/trainer/connectors/data_connector.py:444: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 10 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  rank_zero_warn(

In a normally sized terminal window, the filename can take up a big chunk of space. The user then only sees very little of the actual warning message, and has to scroll horizontally which could be considered annoying.

I experimented with custom formatting. Here are a few versions:

Version 1: Multi-line: Warning type on same line

/Users/adrian/repositories/lightning/src/lightning/pytorch/trainer/connectors/data_connector.py:444: PossibleUserWarning: 
The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider
increasing the value of the `num_workers` argument` (try 10 which is the number of cpus on this
machine) in the `DataLoader` init to improve performance.

Version 2: Multi-line: Warning type on new line

/Users/adrian/repositories/lightning/src/lightning/pytorch/trainer/connectors/data_connector.py:444:
PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a
bottleneck. Consider increasing the value of the `num_workers` argument` (try 10 which is the number
of cpus on this machine) in the `DataLoader` init to improve performance.

Version 3: Multi-line: With indentation

/Users/adrian/repositories/lightning/src/lightning/pytorch/trainer/connectors/data_connector.py:444:
    PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a
    bottleneck. Consider increasing the value of the `num_workers` argument` (try 10 which is the number
    of cpus on this machine) in the `DataLoader` init to improve performance.

Version 4: Single line, but no awkward "rank_zero_only(" on new line

/Users/adrian/repositories/lightning/src/lightning/pytorch/trainer/connectors/data_connector.py:444: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 10 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.

Please let me know whether you find this a good idea, you have any suggestions or if you see any problems with this formatting style.

cc @Borda @carmocca @justusschock @awaelchli

@awaelchli awaelchli marked this pull request as draft August 11, 2023 22:03
@github-actions github-actions bot added the fabric lightning.fabric.Fabric label Aug 11, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Aug 11, 2023

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, lightning, 3.8, 1.11) success
pl-cpu (macOS-11, lightning, 3.9, 1.12) success
pl-cpu (macOS-11, lightning, 3.10, 1.13) success
pl-cpu (macOS-11, lightning, 3.10, 2.0) success
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest) success
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11) success
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest) success
pl-cpu (windows-2022, lightning, 3.8, 1.11) success
pl-cpu (windows-2022, lightning, 3.9, 1.12) success
pl-cpu (windows-2022, lightning, 3.10, 1.13) success
pl-cpu (windows-2022, lightning, 3.10, 2.0) success
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest) success
pl-cpu (macOS-11, pytorch, 3.8, 1.13) success
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13) success
pl-cpu (windows-2022, pytorch, 3.8, 1.13) success
pl-cpu (macOS-12, pytorch, 3.11, 2.0) success
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0) success
pl-cpu (windows-2022, pytorch, 3.11, 2.0) success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
[pytorch-lightning (GPUs) (testing Lightning latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=175018&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e) success
[pytorch-lightning (GPUs) (testing PyTorch latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=175018&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f) success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py.

🟢 pytorch_lightning: Benchmarks
Check ID Status
lightning.Benchmarks success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py.

🟢 fabric: Docs
Check ID Status
docs-make (fabric, doctest) success
docs-make (fabric, html) success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py.

🟢 lightning_fabric: CPU workflow
Check ID Status
fabric-cpu (macOS-11, lightning, 3.8, 1.11) success
fabric-cpu (macOS-11, lightning, 3.9, 1.12) success
fabric-cpu (macOS-11, lightning, 3.10, 1.13) success
fabric-cpu (macOS-11, lightning, 3.10, 2.0) success
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest) success
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11) success
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest) success
fabric-cpu (windows-2022, lightning, 3.8, 1.11) success
fabric-cpu (windows-2022, lightning, 3.9, 1.12) success
fabric-cpu (windows-2022, lightning, 3.10, 1.13) success
fabric-cpu (windows-2022, lightning, 3.10, 2.0) success
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest) success
fabric-cpu (macOS-11, fabric, 3.8, 1.13) success
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13) success
fabric-cpu (windows-2022, fabric, 3.8, 1.13) success
fabric-cpu (macOS-12, fabric, 3.11, 2.0) success
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0) success
fabric-cpu (windows-2022, fabric, 3.11, 2.0) success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py, tests/tests_fabric/utilities/test_warnings.py.

🟢 lightning_fabric: Azure GPU
Check ID Status
[lightning-fabric (GPUs) (testing Fabric latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=175020&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f) success
[lightning-fabric (GPUs) (testing Lightning latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=175020&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e) success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py, tests/tests_fabric/utilities/test_warnings.py.

🟢 mypy
Check ID Status
mypy success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.8) success
install-pkg (ubuntu-22.04, app, 3.11) success
install-pkg (ubuntu-22.04, fabric, 3.8) success
install-pkg (ubuntu-22.04, fabric, 3.11) success
install-pkg (ubuntu-22.04, pytorch, 3.8) success
install-pkg (ubuntu-22.04, pytorch, 3.11) success
install-pkg (ubuntu-22.04, lightning, 3.8) success
install-pkg (ubuntu-22.04, lightning, 3.11) success
install-pkg (ubuntu-22.04, notset, 3.8) success
install-pkg (ubuntu-22.04, notset, 3.11) success
install-pkg (macOS-12, app, 3.8) success
install-pkg (macOS-12, app, 3.11) success
install-pkg (macOS-12, fabric, 3.8) success
install-pkg (macOS-12, fabric, 3.11) success
install-pkg (macOS-12, pytorch, 3.8) success
install-pkg (macOS-12, pytorch, 3.11) success
install-pkg (macOS-12, lightning, 3.8) success
install-pkg (macOS-12, lightning, 3.11) success
install-pkg (macOS-12, notset, 3.8) success
install-pkg (macOS-12, notset, 3.11) success
install-pkg (windows-2022, app, 3.8) success
install-pkg (windows-2022, app, 3.11) success
install-pkg (windows-2022, fabric, 3.8) success
install-pkg (windows-2022, fabric, 3.11) success
install-pkg (windows-2022, pytorch, 3.8) success
install-pkg (windows-2022, pytorch, 3.11) success
install-pkg (windows-2022, lightning, 3.8) success
install-pkg (windows-2022, lightning, 3.11) success
install-pkg (windows-2022, notset, 3.8) success
install-pkg (windows-2022, notset, 3.11) success

These checks are required after the changes to src/lightning/fabric/utilities/warnings.py.


Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

@awaelchli awaelchli force-pushed the feature/better-warning branch from ac34427 to dbc1ded Compare August 11, 2023 22:09
@awaelchli awaelchli added pl Generic label for PyTorch Lightning package fun Staff contributions outside working hours - to differentiate from the "community" label labels Aug 11, 2023
@carmocca
Copy link
Contributor

My vote is (4) - thank you by the way for trying this, didn't know removing the leftover "rank_zero_only(" was possible - because a large percentage of terminals (if not most) will autowrap by default. And those who don't have it configured probably like that their lines do not wrap.

One more suggestion from me would be to remove the warning category. We only use UserWarning (default) or PossibleUserWarning. And the latter was introduced just so people could filter all possible user warnings by a single class. I don't think showing it in the message is particularly useful.

@github-actions github-actions bot added pl Generic label for PyTorch Lightning package and removed pl Generic label for PyTorch Lightning package labels Aug 12, 2023
@awaelchli awaelchli added this to the 2.1 milestone Aug 12, 2023
@awaelchli awaelchli marked this pull request as ready for review August 12, 2023 11:26
@awaelchli awaelchli force-pushed the feature/better-warning branch 2 times, most recently from 4639955 to 31114ed Compare August 15, 2023 14:19
@codecov
Copy link

codecov bot commented Aug 16, 2023

Codecov Report

Merging #18288 (61544e1) into master (fbdbe63) will decrease coverage by 25%.
The diff coverage is 88%.

Additional details and impacted files
@@            Coverage Diff            @@
##           master   #18288     +/-   ##
=========================================
- Coverage      85%      60%    -25%     
=========================================
  Files         427      419      -8     
  Lines       33138    33017    -121     
=========================================
- Hits        28188    19888   -8300     
- Misses       4950    13129   +8179     

@Borda Borda requested a review from carmocca August 16, 2023 15:18
@mergify mergify bot removed the has conflicts label Sep 16, 2023
@awaelchli awaelchli force-pushed the feature/better-warning branch 2 times, most recently from 2a760d7 to 11905b2 Compare September 16, 2023 20:17
@awaelchli awaelchli force-pushed the feature/better-warning branch from ad39540 to 6fde265 Compare September 16, 2023 20:51
@awaelchli awaelchli force-pushed the feature/better-warning branch from 89a6e5d to 2877985 Compare September 18, 2023 21:44
@awaelchli awaelchli force-pushed the feature/better-warning branch from 9f54d75 to d0d6c4b Compare September 18, 2023 21:47
Copy link
Contributor

@carmocca carmocca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome feature

@carmocca carmocca merged commit bc85c5f into master Sep 19, 2023
@carmocca carmocca deleted the feature/better-warning branch September 19, 2023 00:12
@mergify mergify bot added the ready PRs ready to be merged label Sep 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fabric lightning.fabric.Fabric fun Staff contributions outside working hours - to differentiate from the "community" label pl Generic label for PyTorch Lightning package ready PRs ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants