Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a more friendly and useful message when using default config file #1233

Merged
merged 14 commits into from
Jul 21, 2021

Conversation

valeriupredoi
Copy link
Contributor

@valeriupredoi valeriupredoi commented Jul 16, 2021

Description

When there is no user-specified config-user file, the info stdout says Using config file None, which is both untrue and user confusing, even though the actual value is indeed None

Closes ESMValGroup/ESMValTool#2235

Link to documentation:


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.


To help with the number pull requests:

@valeriupredoi valeriupredoi added the UX User experience label Jul 16, 2021
@valeriupredoi valeriupredoi requested review from zklaus and jvegreg July 16, 2021 10:59
@codecov
Copy link

codecov bot commented Jul 16, 2021

Codecov Report

Merging #1233 (93aee71) into main (2a0bb11) will increase coverage by 0.83%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1233      +/-   ##
==========================================
+ Coverage   85.54%   86.37%   +0.83%     
==========================================
  Files         188      188              
  Lines        9150     9153       +3     
==========================================
+ Hits         7827     7906      +79     
+ Misses       1323     1247      -76     
Impacted Files Coverage Δ
esmvalcore/_config/_config.py 88.70% <100.00%> (+0.90%) ⬆️
esmvalcore/_main.py 63.78% <100.00%> (+28.64%) ⬆️
esmvalcore/_recipe_checks.py 82.50% <100.00%> (+0.29%) ⬆️
...smvalcore/experimental/config/_validated_config.py 77.08% <0.00%> (ø)
esmvalcore/_recipe.py 91.37% <0.00%> (+0.12%) ⬆️
esmvalcore/_task.py 61.88% <0.00%> (+4.94%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a0bb11...93aee71. Read the comment docs.

@valeriupredoi
Copy link
Contributor Author

OMG OMG the test coverage decreases by 0.01% 😮 🤣 I like this functionality a lot but is there any way we can set thresholds above which we accept the codecov test to pass? @bouweandela

@zklaus
Copy link

zklaus commented Jul 16, 2021

Why not just improve the coverage?

@jvegreg
Copy link
Contributor

jvegreg commented Jul 16, 2021

Nice touch V! I changed it a bit so we keep the actual path to the config file in the config dictionary, so we can log it directly.

I think is clearer that way and we also do not have to modify the message in the (unlikely) case that we modify the default path

@valeriupredoi
Copy link
Contributor Author

valeriupredoi commented Jul 16, 2021

awesome, cheers Javi! Klaus, I refuse to write tests for print statements, just kidding, I should actually 😁

@zklaus
Copy link

zklaus commented Jul 16, 2021

I mean, the print statement is probably part of a function that does something. As such, it should be covered by tests because that function is covered by tests.

@valeriupredoi
Copy link
Contributor Author

valeriupredoi commented Jul 16, 2021

@jvegasbsc I've added a (rather peasant-y) test that actually calls and runs esmvaltool - it occured to me that a lot of the _main.py functionality is not tested unless one runs the thing by calling the executable. I don't like the fact that I'm writing to cwd and then removing the output, but I can't think of any better option without passing and editing a config file. Is there any way to direct the execution straight to tmp_dir? BTW coverage increases by a hefty 35%

@valeriupredoi
Copy link
Contributor Author

@bouweandela we need to fix this Codecov thingie, man, the latest code changes improve test coverage for _main.py by 35% but the tool fails still? And caches the old result too, it's confusing

@valeriupredoi
Copy link
Contributor Author

@jvegasbsc careful that now there are two R test diags that fail in tests/integration/test_diagnostic_run.py

@jvegreg
Copy link
Contributor

jvegreg commented Jul 20, 2021

I have changed the run test to do not overwrite the default config user and use one in the tmp path and added a new test for the lines that codecov is claiming that now are not covered.

@valeriupredoi, probably you should move the codecov config changes to its own pull request

@jvegreg
Copy link
Contributor

jvegreg commented Jul 20, 2021

I messed a bit too much with this PR to be an independent reviewer. @sloosvel, can you take a look?

@jvegreg jvegreg removed their request for review July 20, 2021 11:41
@jvegreg jvegreg requested a review from sloosvel July 20, 2021 11:41
@valeriupredoi
Copy link
Contributor Author

cheers @jvegasbsc - I took out the codecov changes to the config and cleaned up your bad key that was making the diag test run fail 😁 Heads up that the errors reported by esmvalcore/experimental/config/_validated_config.py during the validation checks (at around lines 40-43) are obscure, we really need a more user-friendly error report from there 👍

@jvegreg
Copy link
Contributor

jvegreg commented Jul 20, 2021

Why not just improve the coverage?

We finally got quite an improvement

@valeriupredoi
Copy link
Contributor Author

Why not just improve the coverage?

We finally got quite an improvement

yesh we did - it is biutiful 😁

Copy link
Contributor

@sloosvel sloosvel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work fine

@valeriupredoi
Copy link
Contributor Author

cheers muchly @sloosvel - @jvegasbsc if you don't want to add more then you can just merge it, mate, I would but that'd be a bit biased 😁

@jvegreg jvegreg changed the title add a more friendly and useful message when using default config file (ESMValTool issue since it deals with user experience) Add a more friendly and useful message when using default config file Jul 21, 2021
@jvegreg
Copy link
Contributor

jvegreg commented Jul 21, 2021

The conda build is failing due to timeout, but that's not our fault (see #1243)

@jvegreg jvegreg merged commit d87679b into main Jul 21, 2021
@jvegreg jvegreg deleted the more_friendly_message_default_config_file branch July 21, 2021 09:36
log_dir = './esmvaltool_output'
log_file = os.path.join(log_dir,
os.listdir(log_dir)[0], 'run', 'main_log.txt')
os.system("rm -r esmvaltool_output")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very unsafe, why not use tmp_path for the output and let pytest clean up?

# default: ~/default_inputpath
# CORDEX: ~/default_inputpath
# Comment out these when using a site-specific path
rootpath:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was commented out because we got the complaint from users that these default settings do not work ever for anyone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request UX User experience
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stdout says "INFO Using config file None" when using default config-user file in $HOME/.esmvaltool
5 participants