Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Feature: XAS Plugin #580

Merged
merged 13 commits into from
Jan 29, 2024
Merged

Add Feature: XAS Plugin #580

merged 13 commits into from
Jan 29, 2024

Conversation

PNOGillespie
Copy link
Contributor

Overview

This PR adds a plugin for calculating X-Ray Absorption (XAS) spectra using the XspectraCrystalWorkChain of AiiDA-QE. Currently this is implemented for K-edge calculations of neutral systems. Calculations of systems with hubbard parameters and magnetic configurations can be made possible, though would require a backend change to AiiDA-QE (see aiidateam/aiida-quantumespresso#969 for relevant PR)
XAS_Plugin_Select

Setting Panel

The Setting panel for the plugin allows users to simply select which element to calculate XAS for, as well as tuning the supercell size to be used in the calculation. This panel also provides the option to change the type of core-hole approximation between FCH (removing the excited electron with tot_charge = 1) and XCH (allowing the excited electron to occupy the conduction band. Since it is still an ongoing subject of debate in the literature, we allow the user to change this value - though recommended values for each element are provided, which will also occupy the default value in the dropdown box.
XAS_Plugin_Setting_Panel
Note that only elements for which appropriate pseudopotentials are available will be presented in the panel. See later section for notes about pseudopotentials.

Results Panel

Overview

The final XAS spectrum is displayed in the Result panel. A dropdown box selects which element to display spectra for. The widget will display both the complete powder XAS for the input structure and the contributions from each symmetrically non-equivalent site. Note that all spectra are normalised with respect to area under the peak by default. Spectra from different sites are scaled in intensity according to the ratio of site multiplicity (number of symmetrically-equivalent sites) to total multiplicity (number of sites for the element).
XAS_Plugin_Result_Panel

Broadening Tools

Broadening effects present in experimental XAS data greatly affect the appearance of the final spectrum (see Bunau & Calandra (2013)) and so post-procesing of spectra to include these effects (which cannot be computed ab-initio with DFT) is often used in published works using DFT-derived spectra. We therefore provide a set of broadening tools for the final spectrum. The tools apply one of two Lorentzian broadening schemes (toggled by a checkbox widget):

  1. Constant broadening applied to the entire spectrum.
  2. Variable-energy broadening using a starting value ($\Gamma_{hole}$), maximum value ($\Gamma_{max}$, and a mid-point energy ($E_{Center}$).

The 1st scheme uses just $\Gamma_{hole}$ (which refers to the natural linewidth of the core level) to define the Lorentzian parameter. The 2nd scheme uses each parameter to define an s-curve function with an inflection point at $E_{Center}$. The broadening scheme is applied to all spectra on the plot simultaneously. Note that when the spectrum is downloaded, the plotted data on the Plotly widget (including broadening) is what is written to the CSV file - not the raw data. To view the raw data, the user can simply set $\Gamma_{hole}$ to 0.

Pseudopotentials

Calculations of XAS using QE require a pseudopotential to define the electron configuration of the final state of the excitation for the absorbing atom. In addition, a radial coordinate plot of the initial state (e.g. the 1s state of the absorbing atom in a K-edge calculation) is needed by XSpectra. To provide both of these, including a relevant pseudopotential for the ground-state of the absorbing element (and from which we generate the initial state wavefunction) a small pseudopotential library is provided with the plugin. This library is taken from another GitHub repository (https://github.com/PNOGillespie/Core_Level_Spectra_Pseudos) which is a small collection of pseudos which we have used already in XAS calculations. This repo has an archive file for each XC functional (currently only PBE) and a "Pseudo Table of Contents" (pseudo_toc.yaml) file to record the filenames and directories for each pseudo and associated core wavefunction file. The plugin uses this YAML file to import files, check for missing files, load the correct nodes for calculation, and set the default core-hole approximation. The only requirement for maintainers is just to copy the YAML file from Core_Level_Spectra_Pseudos into the plugin directory in order to update the local pseudo archive (which is done automatically on startup if the archive file is missing or there are pseudos missing which should be there).

Adds a working version of a plugin to calculate XANES spectra
using the XspectraCrystalWorkChain of AiiDA-QE

This commit requires a locally-situated copy of the core-hole
pseudopotentials required for XSpectra to function, which is obtained
from https://github.com/PNOGillespie/Core_Level_Spectra_Pseudos and
downloaded and installed upon first activation of AiiDALab-QE
Refactors the system for importing core-hole pseudopotentials from
using groups to using a `yaml` file to define which pseudos should
be present in the ch pseudos archive. Greatly simplifies the logic
for checking for the presence of pseudopotentials and populating
lists of available pseudos.

Yaml file `pseudo_toc` is obtained from the Core_Level_Spectra_Pseudos
archive (https://github.com/PNOGillespie/Core_Level_Spectra_Pseudos)
and must be updated on both the archive and the plugin when new
pseudos are available on the CLS archive.

Also adds a step in `workchain.py` to check for mismatches in
which pseudos should be present and re-download the archive file
if there is a mismatch. Added in order to check for inconsistencies
and to automatically update the archive if `pseudo_toc.yaml` is
updated with new pseudos.
Migrates the directory used for core-level spectra pseudos to
`~/.local/lib/cls_pseudos` in order to allow integration tests
to pass.
Adds a range of improvements and fixes to the plugin.
Fixes:
* Fixes an issue where the `Setting` panel of the plugin would
cause a crash due to a variable being re-defined during runtime.
* Adds a `.close()` statement to the end of the HTTP request when
accessing the pseudo archive file for download.
Additions:
* The app now retrieves all contributions to the final spectrum for the
selected element and plots each together. The spectra are aligned in terms of
energy in the same way as the `XspectraCrystalWorkChain` uses to generate the
final spectrum. Each contributing spectrum is scaled according to the ratio of
site multiplicity to total multiplicity. The provided broadening tools will apply
to all spectra on the widget simultaneously.
* Adds a button to the `Result` panel to download a CSV file of the spectra
presented on the Plotly widget. This will download a file titled
`{element}_XAS_Spectra.csv` which contains all spectra on the widget (total
spectrum and all site contributions). For sites with a multiplicity ratio
different from 1, the weighted and unweighted spectra are printed in separate
columns.
Copy link

codecov bot commented Dec 13, 2023

Codecov Report

Attention: 204 lines in your changes are missing coverage. Please review.

Comparison is base (8ad9d9c) 80.91% compared to head (b0a80ac) 77.55%.

Files Patch % Lines
src/aiidalab_qe/plugins/xas/result.py 10.65% 176 Missing ⚠️
src/aiidalab_qe/plugins/xas/workchain.py 34.28% 23 Missing ⚠️
src/aiidalab_qe/plugins/xas/setting.py 96.40% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #580      +/-   ##
==========================================
- Coverage   80.91%   77.55%   -3.36%     
==========================================
  Files          49       54       +5     
  Lines        3426     3831     +405     
==========================================
+ Hits         2772     2971     +199     
- Misses        654      860     +206     
Flag Coverage Δ
python-3.10 77.55% <49.25%> (?)
python-3.8 77.58% <49.25%> (-3.33%) ⬇️
python-3.9 77.58% <49.25%> (-3.33%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@PNOGillespie PNOGillespie changed the title Xas plugin Add Feature: XAS Plugin Dec 13, 2023
unkcpz
unkcpz previously requested changes Dec 15, 2023
Copy link
Member

@unkcpz unkcpz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool! Welcome XAS to join the QeApp family.
I give a broad look with two comments, @superstar54 is capable of giving it a detailed look.

Comment on lines +6 to +11
C: C.pbe-n-kjgipaw_psl.1.0.0.UPF
Cu: Cu.pbe-n-van_gipaw.UPF
F: F.pbe-gipaw_kj_no_hole.UPF
Li: Li.pbe-s-rrkjus-gipaw.UPF
O: O.pbe-n-kjpaw_gipaw.UPF
Si: Si.pbe-van_gipaw.UPF
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to slow down the progress, is that possible to integrate this to aiida-pseudo?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I will say for this that the solution being used here is not necessarily intended as a permanent solution to providing pseudopotentials for these types of calculations. In principle, integrating some sort of family in aiida-pseudo for core-hole pseudopotentials is the ideal long-term objective here - since then the procedure for assigning pseudos obviously follows that of other plugins.

The two issues I see with integrating the current library with aiida-pseudo are:

  1. Some sort of new Class or other new feature will need to be made for aiida-pseudo in order to work with these types of pseudopotentials: for instance to ensure the correct core wavefunction data file is matched to the correct GIPAW pseudo, also that the core-hole and GIPAW pseudos are matched correctly.
  2. The current library is a stub which covers only some elements, therefore potentially not enough to make it worth pursuing with just what's there at the moment.

I would be interested in your thoughts on this in any case @unkcpz.

Comment on lines +17 to +20
"""Download button with dynamic content
The content is generated using a callback when the button is clicked.
Modified from responses to https://stackoverflow.com/questions/61708701/how-to-download-a-file-using-ipywidget-button#62641240
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try it using the AWB version and see how that goes. In principle, it looks like it should be fine since the file for the output spectra is created outside of the widget, but I will let you know if there are any issues with using it here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, @PNOGillespie I think what you did are correct, the export widget is for exporting the aiida archive, we don't yet have the generic widget for export the file.
Can you try to it to aiidalab-qe/src/aiidalab_qe/common/, @AndresOrtegaGuerrero also implement a similar widget.

Copy link
Contributor Author

@PNOGillespie PNOGillespie Dec 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure @unkcpz, I will work on using the export widget with the plugin, though I'm currently only a couple of days from vacation, so maybe in January at this rate 😆

I had intended it to show explicitly (for the user) that it would download a CSV file - also that the filename be related to the element selected to be displayed (e.g. O_K-edge_XAS.csv for spectra of oxygen). Does the widget allow for the filename or description text to be specified? @AndresOrtegaGuerrero

I don't mind working on changes to export in order to make this happen, if that's ok with everyone.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PNOGillespie i think @unkcpz refers to the BandsWidget PR #581 , you can adapt the export (code) to your needs, like the filename name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @unkcpz & @AndresOrtegaGuerrero, I've run some tests with the DownloadButton widget and compared it to the SpectrumDownloadButton widget I made for this plugin. I can get all the same functionality out of DownloadButton as I did for my own one, though the issue I notice is that the Plotly viewer runs noticeably slower when working with the DownloadButton.

The reason, I'm guessing, is that the DownloadButton needs the file's contents (payload) to be a bytes object, so each time the spectrum is re-drawn (e.g. if the spectrum broadening or element selection is changed) it needs to encode a new bytes object, which takes extra time (probably even more for spectra composed of multiple different absorbing atoms). The SpectrumDownloadButton doesn't have this issue because encoding is done during __on_click and so the input can just be a str object.

I do like the idea of a generalised DownloadButton widget (particularly as there's only really one way to do make this work anyway) but I think it needs some encoding methods implemented to enhance the performance.

Copy link
Member

@superstar54 superstar54 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PNOGillespie thanks for the work!

I did some test calculations. Here are some comments on the workchain part.

src/aiidalab_qe/plugins/xas/workchain.py Outdated Show resolved Hide resolved
src/aiidalab_qe/plugins/xas/workchain.py Show resolved Hide resolved
src/aiidalab_qe/plugins/xas/workchain.py Outdated Show resolved Hide resolved
PNOGillespie and others added 2 commits January 24, 2024 15:07
Enables the `supercell_min_parameter` setting for XAS calculations

Also removes various extraneous lines of code which were commented
out, had typos, or were requested for deletion.
Copy link
Member

@superstar54 superstar54 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @PNOGillespie, I see that you checked and downloaded the pseudo in the workchain.py part. I suggest moving them to the setting.py. Because the pseudo information is used in the setting panel. Of course, you can do this in another PR.

The PR will be good to me when you make the changes.

For the download button, let Andres finish his, and we can discuss which one to go with. You can update it in the future PR.

self.on_click(self.__on_click)

def __on_click(self, b):
if self.contents() is None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You pass Non to contents when creating the download_data button

Suggested change
if self.contents() is None:
if self.contents is None:

<h4>Element and Core-Hole Treatment Setting.</h4></div>"""
)

# TODO: The element selection should lock the "Confirm" button if no elements have been
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you open a issue? so that we can work on this in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll open one once this PR is concluded, no problem.

}
return parameters

def set_panel_value(self, input_dict):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the element_and_ch_treatment_options is not set properly. I think you need to read the
core_hole_treatments, and elements_list from then input_dict, and set the widget values.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this did confuse me a little on review, but element_and_ch_treatment_options is indeed not set, however it also doesn't do anything. The core_hole_treatments and elements_list however are set correctly - I tested it just now - but that's defined by the code in _update_element_selection_panel() (lines 166+).

The short answer to this is that we can just delete all of the code in lines 141-159. I can discuss this in more detail separately if you would like.

Comment on lines 141 to 159
ch_pseudos = self.core_hole_pseudos
structure = self.input_structure
available_elements = [k for k in ch_pseudos]
elements_to_select = sorted(
[
kind.symbol
for kind in structure.kinds
if kind.symbol in available_elements
]
)

element_and_ch_treatment_options = {}
for element in elements_to_select:
if element in xch_elements:
element_and_ch_treatment_options[element] = "xch_smear"
else:
element_and_ch_treatment_options[element] = "full"

self.element_and_ch_treatment_options = element_and_ch_treatment_options
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @superstar54. Does this seem like a reasonable way to address your previous comment?

I suggest this because the logic to define the initial values for the element_and_ch_treatment panel has already been defined in _update_element_select_panel(). This works fine with my own testing and sets the right behaviour for set_panel_value().

Suggested change
ch_pseudos = self.core_hole_pseudos
structure = self.input_structure
available_elements = [k for k in ch_pseudos]
elements_to_select = sorted(
[
kind.symbol
for kind in structure.kinds
if kind.symbol in available_elements
]
)
element_and_ch_treatment_options = {}
for element in elements_to_select:
if element in xch_elements:
element_and_ch_treatment_options[element] = "xch_smear"
else:
element_and_ch_treatment_options[element] = "full"
self.element_and_ch_treatment_options = element_and_ch_treatment_options
self._update_element_select_panel()

PNOGillespie and others added 4 commits January 26, 2024 16:17
Changes requested changes for PR aiidalab#580:

* Moves the loading of pseudos and core wavefunction
data to `setting.py`. `setting.py` now collects labels
for pseudos and core_wfc_data, while `workchain.py`
loads the nodes from the labels.
* Replaces `set_panel_value()` content with call to
`self._update_element_select_panel()`.
* Fixes typos and removes redundant code noted in
latest review.
@superstar54
Copy link
Member

The codecov decreased because there is no test on the workchain and its result. We will come to this when we resolve #533.

@superstar54 superstar54 self-requested a review January 29, 2024 21:21
@unkcpz
Copy link
Member

unkcpz commented Jan 29, 2024

The codecov decreased because there is no test on the workchain and its result. We will come to this when we resolve #533.

Thanks for mentioning this. I'd suggest we iron the test first.

@superstar54 superstar54 dismissed unkcpz’s stale review January 29, 2024 21:33

We leave the pseudopotential setup and the download button to the feature PR.

@superstar54 superstar54 merged commit 1950c34 into aiidalab:main Jan 29, 2024
16 of 18 checks passed
@unkcpz
Copy link
Member

unkcpz commented Jan 30, 2024

We leave the pseudopotential setup and the download button to the feature PR.

As we talked over the phone, please open issues for these isuses that not addressed. And we should try to do it as soon as possible, otherwise they will never happened. @superstar54 @PNOGillespie

@unkcpz
Copy link
Member

unkcpz commented Jan 30, 2024

Also a mild reminder that try not do the hush merge next time. If you have DDL then we need to plan things ahead and make things happened under the plan. The integration test is failed so please try to fix it by this week @superstar54 @PNOGillespie

@unkcpz
Copy link
Member

unkcpz commented Jan 30, 2024

In fact there is a crucial break in the PR, the integration test shows:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/tmp/ipykernel_1713/2189467020.py in <cell line: 17>()
     15 from jinja2 import Environment
     16 
---> 17 from aiidalab_qe.app import App, static
     18 from aiidalab_qe.version import __version__

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/app/__init__.py in <module>
      1 """Package for the AiiDAlab QE app."""
      2 
----> 3 from .main import App
      4 
      5 __all__ = [

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/app/main.py in <module>
     12 from aiidalab_qe.app.result import ViewQeAppWorkChainStatusAndResultsStep
     13 from aiidalab_qe.app.structure import StructureSelectionStep
---> 14 from aiidalab_qe.app.submission import SubmitQeAppWorkChainStep
     15 from aiidalab_qe.common import QeAppWorkChainSelector
     16 

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/app/submission/__init__.py in <module>
     20 from aiidalab_qe.common.setup_codes import QESetupWidget
     21 from aiidalab_qe.common.setup_pseudos import PseudosInstallWidget
---> 22 from aiidalab_qe.workflows import QeAppWorkChain
     23 
     24 from .resource import ParallelizationSettings, ResourceSelectionWidget

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/workflows/__init__.py in <module>
     40 
     41 
---> 42 plugin_entries = get_entry_items("aiidalab_qe.properties", "workchain")
     43 
     44 

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/workflows/__init__.py in get_entry_items(entry_point_name, item_name)
     32 # load entry point items
     33 def get_entry_items(entry_point_name, item_name="workchain"):
---> 34     entries = get_entries(entry_point_name)
     35     return {
     36         name: entry_point.get(item_name)

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/workflows/__init__.py in get_entries(entry_point_name)
     25     entries = {}
     26     for entry_point in entry_points().get(entry_point_name, []):
---> 27         entries[entry_point.name] = entry_point.load()
     28 
     29     return entries

/opt/conda/lib/python3.9/site-packages/importlib_metadata/__init__.py in load(self)
    205         """
    206         match = self.pattern.match(self.value)
--> 207         module = import_module(match.group('module'))
    208         attrs = filter(None, (match.group('attr') or '').split('.'))
    209         return functools.reduce(getattr, attrs, module)

/opt/conda/lib/python3.9/importlib/__init__.py in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/plugins/xas/__init__.py in <module>
      8 
      9 from .result import Result
---> 10 from .setting import Setting
     11 from .workchain import workchain_and_builder
     12 

/opt/conda/lib/python3.9/site-packages/aiidalab_qe/plugins/xas/setting.py in <module>
     17 from aiidalab_qe.plugins import xas as xas_folder
     18 
---> 19 PSEUDO_TOC = yaml.safe_load(resources.read_text(xas_folder, "pseudo_toc.yaml"))
     20 pseudo_data_dict = PSEUDO_TOC["pseudos"]
     21 xch_elements = PSEUDO_TOC["xas_xch_elements"]

/opt/conda/lib/python3.9/importlib/resources.py in read_text(package, resource, encoding, errors)
    137     bytes.decode().
    138     """
--> 139     with open_text(package, resource, encoding, errors) as fp:
    140         return fp.read()
    141 

/opt/conda/lib/python3.9/importlib/resources.py in open_text(package, resource, encoding, errors)
    119     """Return a file-like object opened for text reading of the resource."""
    120     return TextIOWrapper(
--> 121         open_binary(package, resource), encoding=encoding, errors=errors)
    122 
    123 

/opt/conda/lib/python3.9/importlib/resources.py in open_binary(package, resource)
     89     reader = _get_resource_reader(package)
     90     if reader is not None:
---> 91         return reader.open_resource(resource)
     92     absolute_package_path = os.path.abspath(
     93         package.__spec__.origin or 'non-existent file')

/opt/conda/lib/python3.9/importlib/_bootstrap_external.py in open_resource(self, resource)

FileNotFoundError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.9/site-packages/aiidalab_qe/plugins/xas/pseudo_toc.yaml'

@unkcpz
Copy link
Member

unkcpz commented Jan 30, 2024

You can reproduce it by start the container docker run --rm -it -p 8888:8888 aiidalab/qe:edge and go to the app to see the error.

@superstar54
Copy link
Member

@unkcpz Sorry for the bug. It should be fixed in #609 .

@unkcpz
Copy link
Member

unkcpz commented Jan 30, 2024

No worries, I add the integration test to the required test. It is quite robust and can give useful information, so we need to put an eye on it next time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants