Template resources setting

[unkcpz]

Supersede #483, fixes #482
also fixes aiidalab/aiidalab-qe#240

• UI re-organized by having resource database select in quick setup and having a switch for only detailed setup
• Refactoring and decouple quick setup
• Decouple detailed setup
• Unit test for resource widgets.
• Help text for detailed setup, when only detailed setup don't use tab widget.
• layout reorganized for quick setup. Having account/password aligned as normal login UI.
• Check XXX 001
• Actual template implementation for computer and code, check WIP: using template to setup the computer/code #483 for the demo.
• Check all XXX label 002 is cleaned.
• glitches fixes:
  • reset and change computer, the template of code not show up.
  • the help text not disappeared after select new code without template fields.
  • database widget reset should alse reset the option of sub-widgets

Feedbacks from @giovannipizzi after demo in the aiidalab meeting, can add in the template a section "meta" for information like

• which variables can be set for template fields. ("pw", "ph" ... the valid binary name for QE)
• The mapping from the template variable to the name prompted showing in the widget. ("slurm_account" -> "Slurm account").

More tasks after @superstar54's test on CSCS machine.

• Since CSCS needs 2FA now, it makes no sense to ask for password (maybe they didn't deprecate password for all users?). The solution then become using aiidalab-mfa-cscs, and having a template metadata field that can display the information.
• default value read and check with option list
• Suppress entry points not found issue when it is a template
• The template should be valid if the entry point matches with the template.
• The template field can continue to vary, means can reset in place.

More tasks after discuss with @yakutovicha.

• The username now can be template for the computer_configure. I think this is a very good design decision, we were not able to put username which is a mandatory field for verdi computer configure since we don't want the code/resoruce registry contain specific information. Now thanks to the flexible template mechanism this information can be asked without specify the final value in the registry setup file.
• The detail setup has different UI layout from quick_setup but share the same traits, combine into one with a toggle to switch the layout can be a better UI design.

Notes for myself:

• Before merge, instead of relying on the remote aiida-resource-registry database, mock one as fixture locally but validate it with the remote schema, this give us more flexibility to and robustness to run unit test.
• Put template variables specification under metadata.template_variables?
• Adjust UI so the box of detailed setup is align good with description
• Pass the notebook tests.

[codecov]

Codecov Report

Attention: 60 lines in your changes are missing coverage. Please review.

Comparison is base (5b303df) 79.92% compared to head (4060723) 82.79%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #511      +/-   ##
==========================================
+ Coverage   79.92%   82.79%   +2.87%     
==========================================
  Files          27       27              
  Lines        3815     4313     +498     
==========================================
+ Hits         3049     3571     +522     
+ Misses        766      742      -24     

Flag	Coverage Δ
python-3.10	82.79% <92.05%> (+2.87%)	⬆️
python-3.8	82.83% <92.05%> (+2.87%)	⬆️
python-3.9	82.83% <92.05%> (+2.87%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
aiidalab_widgets_base/utils/__init__.py	75.22% <100.00%> (+12.72%)	⬆️
tests/test_databases.py	100.00% <100.00%> (ø)
tests/test_computational_resources.py	99.70% <99.60%> (-0.30%)	⬇️
aiidalab_widgets_base/databases.py	93.71% <96.80%> (-0.41%)	⬇️
aiidalab_widgets_base/computational_resources.py	79.87% <84.94%> (+10.02%)	⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[danielhollas]

@unkcpz I would recommend to merge with master branch, I made some fixes in #502 and #508

[unkcpz]

Thanks @danielhollas, I am thinking about it as well. I rebase it.

[unkcpz]

Hi @superstar54, @yakutovicha, this is ready for review, let me know how you want me to rebase it so you can review it more easily. It is currently every task has its own commit. I can rebase it to:

• Refactoring the Computational resource widget by moving quick_setup/detailed_setup widgets out with adding unit tests.
• New computational resource database widget with adding uint tests.
• Implement the template widget and integrate into quicksetup with unit tests.
• moving the widgets into modules as I did for QeApp in Adapt modules and files layout for plugin implementation aiidalab-qe#439 (Some module files , especially computational_resources.py quite large, I think it is better to move some widgets to files, this is not included in this PR, not sure if we want to have this or not.)

[superstar54]

Hi @unkcpz thanks for the work. the UI looks more organized than before. I did a test using Quick Setup, but got an error,

When I checked the code, I found there was a projwfc-7.2@daint-mc code, which I did select to setup.

[superstar54]

In the Quick Setup UI, it sets up both the computer and code, but the Label only shows daint-mc, which may make the user a little confused.

[unkcpz]

In the Quick Setup UI, it sets up both the computer and code, but the Label only shows daint-mc, which may make the user a little confused.

Good point! This is read from the template, it is the people who upload the template can set the name for this see https://aiidateam.github.io/aiida-resource-registry/database.json, I change it to "Computer Label"

[unkcpz]

Hi @superstar54, could you try again? I find a bug in my code and it is now fixed see f75c79d

[superstar54]

Hi @superstar54, could you try again? I find a bug in my code and it is now fixed see f75c79d

The error still exists. And there is a new error now.

Request for URL https://unkcpz.github.io/aiida-resource-registry/database.json failed; using cached response
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/requests_cache/session.py", line 231, in _resend_and_ignore
    response.raise_for_status()
  File "/opt/conda/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://unkcpz.github.io/aiida-resource-registry/database.json

[unkcpz]

For the new error, it is because I move the aiida-resource-registry, and the update in the pr is in the latest commit.
But I have no idea how to fix the previous error. Let's meet in person and debug together when you are in the office.

[unkcpz]

As discussed with @yakutovicha on Zoom, there are the following things need to be polished:

• computer label not updated with the default value
• The slurm_partition and slurm_account in the prepend_text now in principle should be able to be updated immediately after the template with default value is loaded.
• Try to use dlink instead of observe for _on_template_variables_* callbacks.

[unkcpz]

Hi @yakutovicha, I made all the changes as we discussed. You are right that for the weird behavior where the template placeholder not update, the codes are already there. Problem was when the default value does not exist for a template string, the _fill_tempalte function raise the exception and exits the for loop and following computer_setup are skipped.

For using the dlink, it is feasible for the code_setup and the test passed directly. But for the computer_setup_and_configure, I can not make such a change because for the template widget the trait filled_template is computer_setup and computer_configure, while for the resource widget the trait is computer_setup_and_configure. The transform is not working here because what is needed here is to dlink from computer_setup -> self.computer_setup_and_configure["setup"].

[yakutovicha]

@unkcpz thanks.

Unfortunately, two things that we discussed do not work:

• Template values are updated only when all fields are present.
• When I erased the value from the "Slurm account", it remained the same and did not return to {{ slurm_account }}

Looking at the code I found several issues too, let's try to fix them.

[yakutovicha]

As discussed with @unkcpz separately, I will do the remaining changes.

[unkcpz]

Hi @yakutovicha, I fix all the tests and try my best to clear the flow of the template class. Please have a "final" (:crossed_fingers:) look.

[yakutovicha]

[danielhollas]

Congrats @unkcpz, this is a cool feature! 👏