Fix single det + WFOpt

[ye-luo]

Proposed changes

Fix #2496, both cases
The bug affected single det + WF optimization. When the optimizer reset WF, the single det was not reset clean.
A safeguard is introduced to check recomputed variance against the earlier VMC variance.

The bug affects NLPP evaluation in WFOpt only when the first electron in NLPP radius needing NLPP evaluation is the same as the last electron in NLPP radius of the previous sample. This can occur only when the system has very few electrons and the electron density is low.

This PR can be reviewed independent of #2682 but be sure to merge after #2682

What type(s) of changes does this code introduce?

• Bugfix

Does this introduce a breaking change?

• No

What systems has this change been tested on?

Bora

Checklist

• Yes. This PR is up to date with current the current state of 'develop'
• Yes. Code added or changed in the PR has been clang-formatted

[prckent]

Thanks Ye. Assuming this fixes all the issues, do you have any opinions as to how to avoid mistakes like this in future?

[ye-luo]

I tried to make a precise and trimmed reproducer but it was not successful. So I put this approximate safeguard which is more universally valid like our NaN checks. Apart from what developers adding more tests or checks, I'm wondering if we can build more diagnosis on the statistics in the code or workflow tools to help catching issues in the code.

[prckent]

Does the new check catch both reported errors?

We are clearly lacking in deterministic optimizer checks for different wavefunctions.

[ye-luo]

Does the new check catch both reported errors?

We are clearly lacking in deterministic optimizer checks for different wavefunctions.

Yes. The added safeguard traps the reported bugs.

[prckent]

Adding internal checks is very worthwhile, particularly between the optimizer and the vmc driver where there is a big handoff.