Add sofk estimator test and manual entry about how estimator works #764

Paul-St-Young · 2018-04-12T23:23:47Z

This PR follows up on #756 but only introduce a test and manual updates

Documenting estimator manager has turned out to be a lot more involved than I expected.
I decided to upload the test examples first to get the discussion started.

The new tests can be ran using ctest -R estimator-sofk. I added two SkEstimator estimators per run. One outputs to scalar.dat while the other outputs to stat.h5. DMC is done using a very large time step and 1 step per block. This is a severe test of the ensemble average procedure because both walker weight and walker number undergo large fluctuation.

In the Python script check_collectables_h5dat, I get the S(k) value at the largest k vector and compare between scalar.dat and stat.h5. S(k->large) should fluctuation around 1.0 in both cases. Currently, scalar.dat fluctuates around 1.0 but stat.h5 does not.

add tests to check consistency between scalar.dat and stat.h5 output paths

qmc-robot · 2018-04-12T23:25:01Z

Can one of the maintainers verify this patch?

qmc-robot · 2018-04-12T23:25:02Z

Can one of the maintainers verify this patch?

Paul-St-Young · 2018-04-12T23:26:47Z

@markdewing is there an easy way to pass extra arguments to my Python script using the CMake macro SIMPLE_RUN_AND_CHECK?

I currently copied and pasted the same script for pbyp and allp, changing the "prefix" variable only.

Paul-St-Young · 2018-04-12T23:28:42Z

Regarding the actual bug. I think Collectables is zeroed out at every step, but it may not be populated if all moves are rejected in advanceWalkers.

markdewing · 2018-04-13T04:46:14Z

okay to test

markdewing · 2018-04-13T05:31:11Z

@Paul-St-Young It should be straightforward to add arguments with some changes to the SIMPLE_RUN_AND_CHECK function. Use the ${ARGN} variable to get extra arguments to that function, and pass it the add_test function after ${check_cmd}

It think it should be enough to change the add_test call to

add_test(NAME "${test_name}"
    COMMAND "${check_cmd}" ${ARGN}
    WORKING_DIRECTORY "${work_dir}")

WIth this, any extra arguments given to SIMPLE_RUN_AND_CHECK will be passed as extra arguments to the check script

Paul-St-Young · 2018-04-13T19:44:14Z

Thanks @markdewing! You 1-line solution worked beautifully.

Paul-St-Young · 2018-04-13T19:47:36Z

I added documentation on the current estimator collection procedure. I think there are improvements to be made, but my understanding is that @ye-luo is already working on it.

I will work to fix the current bug within the existing design.

markdewing · 2018-04-13T21:37:22Z

manual/estimator_manager.tex

+\subsection{Class Relations}
+A large number of classes are involved in the estimator collection process. They often have misleading class name or method name. Document gotchas in the following list:
+\begin{enumerate}
+\item \verb|EstimatorManager| is an unused copy of \verb|EstimatorManagerBase|. \verb|EstimatorManagerBase| is the class used in the QMC drivers. (PR \#371 explains this)


Can EstimatorManager.cpp and .h be deleted? (See #371). That would address this point.

Please do not do the deletion of the file but just don't mention it in the tex. I will soon PR some changes including the deletion.

OK, I will take out this item from the doc

markdewing · 2018-04-13T22:21:24Z

manual/estimator_manager.tex

+
+Two ensemble-level data structures \verb|ParticleSet::Properties| and \verb|::Collectables| serve as intermediaries between evaluate classes and output classes to scalar.dat and stat.h5. \verb|Properties| appears in both scalar.dat and stat.h5, whereas \verb|Collectables| appears only in stat.h5. \verb|Properties| is overwritten by \verb|QMCHamiltonian::Observables| at the end of each step. \verb|QMCHamiltonian::Observables| is filled upon call to \verb|QMCHamiltonian::evaluate| and \verb|::auxHevaluate|. \verb|Collectables| is zeroed at the beginning of each step and accumulated upon call to \verb|::auxHevaluate|.
+
+Data are outputted to scalar.dat in 4 stages: evaluate, load, unload, and collect. In the evaluate stage, \verb|QMCHamiltonian::Observables| is populated by a list of \verb|QMCHamiltonianBase|. In the load stage, \verb|QMCHamiltonian::Observables| is transfered to \verb|Properties| by \verb|QMCDriver|. In the unload stage, \verb|Properties| is copied to \verb|LocalEnergyEstimator::scalars|. In the collect stage, \verb|LocalEnergyEstimator::scalars| is block-averaged to \verb|EstimatorManagerBase|\\ \verb|::AverageCache| and dumped to file. For \verb|Collectables|, the evaluate and load stages are combined in a call to \verb|QMCHamiltonian::auxHevaluate|. In the unload stage, \verb|Collectables| is copied to \verb|CollectablesEstimator::scalars|. In the collect stage, \verb|CollectablesEstimator|\\ \verb|::scalars| is block-averaged to \verb|EstimatorManagerBase::AverageCache| and dumped to file.


This paragraph should go before the detailed description of each stage, as an overview.

Agreed. I wrote this summary last, but it would serve as a good overview.

markdewing · 2018-04-13T22:28:30Z

manual/estimator_manager.tex

+\begin{enumerate}
+\item \verb|EstimatorManager| is an unused copy of \verb|EstimatorManagerBase|. \verb|EstimatorManagerBase| is the class used in the QMC drivers. (PR \#371 explains this)
+\item \verb|EstimatorManagerBase::Estimators| is completely different from \verb|QMCDriver::Estimators|, which is subtly different from \verb|QMCHamiltonianBase::Estimators|. The first is a list of pointers to \verb|ScalarEstimatorBase|. The second is the master estimator (one per MPI group). The third is the slave estimator that exists one per OpenMP thread.
+\item \verb|QMCHamiltonian| is NOT a parent class of \verb|QMCHamiltonianBase|. Instead, \verb|QMCHamiltonian| owns two lists of \verb|QMCHamiltonianBase| named \verb|H| and \verb|auxH|.


Based on the names, I would (incorrectly) expect QMCHamiltonian to be derived from QMCHamiltonianBase. It would be good to say what these are (or if they are defined in a previous section). QMCHamiltonianBase is the base class for individual Hamiltonian pieces, and QMCHamiltonian is the container for all of them. H is for Hamiltonian elements that contribute to the energy, and auxH is for elements that do not. (Someone else can comment if this description is wrong?)

This is correct. A "HamiltonianBase" element is really a component of the Hamiltonian for ones that contribute to the energy and just an observable/operator otherwise. The common denominator for HamiltonianBase is that each derived instance represents a quantum mechanical observable/operator.

markdewing · 2018-04-13T22:35:08Z

manual/estimator_manager.tex

+\item \verb|EstimatorManager| is an unused copy of \verb|EstimatorManagerBase|. \verb|EstimatorManagerBase| is the class used in the QMC drivers. (PR \#371 explains this)
+\item \verb|EstimatorManagerBase::Estimators| is completely different from \verb|QMCDriver::Estimators|, which is subtly different from \verb|QMCHamiltonianBase::Estimators|. The first is a list of pointers to \verb|ScalarEstimatorBase|. The second is the master estimator (one per MPI group). The third is the slave estimator that exists one per OpenMP thread.
+\item \verb|QMCHamiltonian| is NOT a parent class of \verb|QMCHamiltonianBase|. Instead, \verb|QMCHamiltonian| owns two lists of \verb|QMCHamiltonianBase| named \verb|H| and \verb|auxH|.
+\item \verb|QMCDriver::H| is NOT the same as \verb|QMCHamiltonian::H|. The first is a pointer to a \verb|QMCHamiltonian|. \verb|QMCHamiltonian::H| is a list.


This could subsumed into the previous item by pointing out the QMCDriver::H is of type QMCHamiltonian (and QMCHamiltonian::H was defined in that point)

markdewing · 2018-04-14T04:02:57Z

manual/estimator_manager.tex

+A large number of classes are involved in the estimator collection process. They often have misleading class name or method name. Document gotchas in the following list:
+\begin{enumerate}
+\item \verb|EstimatorManager| is an unused copy of \verb|EstimatorManagerBase|. \verb|EstimatorManagerBase| is the class used in the QMC drivers. (PR \#371 explains this)
+\item \verb|EstimatorManagerBase::Estimators| is completely different from \verb|QMCDriver::Estimators|, which is subtly different from \verb|QMCHamiltonianBase::Estimators|. The first is a list of pointers to \verb|ScalarEstimatorBase|. The second is the master estimator (one per MPI group). The third is the slave estimator that exists one per OpenMP thread.


It sounds like one big difference between these is related to parallelization - MPI and OpenMP? It seems like it would be helpful to describe how the parallelization structure drives some of structure of the estimator accumulation. Also, how does this structure affect the weighted summation in E_VMC and E_DMC?

markdewing · 2018-04-14T04:24:16Z

manual/estimator_manager.tex

+
+In a DMC simulation, the weight of each walker is different and may change from step to step. Further, the ensemble size varies from step to step. Therefore, eq.~(\ref{eq:estimator}) simplifies to
+\begin{align}
+E_{DMC}[O] = \frac{1}{N_{step}} \sum_{s} \left\{ \left(\sum_e w_{s,e} O(\bs{R}_{s,e})  \right) / \left( \sum \limits_{e} w_{s,e} \right)  \right\}.


How do the differences in the scalar.dat and stat.h5 come about? Are there differences in how this weighted average gets grouped and implemented?

The change from E[O] to E_DMC[O] looks like an approximation, not an identity (?) To be completely correct, there should be a weight for each ensemble average as well ( ensemble weight = sum_e w_{s,e} / sum of all weights). The bias will be reduced if the ensemble weights do not vary much (limit of large ensemble or small timesteps).

prckent · 2018-11-27T18:42:39Z

This PR has not been updated in some while. There are a few outstanding questions that need answering before merging and some conflicts to fix. Should we leave this PR open?

ye-luo · 2018-11-27T19:07:00Z

I fixed the merge conflict. This PR seems not solving our issue but mostly documentation, it can be merged.

ye-luo

Better to have this PR merged. We will revise the manual entry as we refactor estimator manager.

qmc-robot · 2018-12-17T19:55:14Z

Can one of the admins verify this patch?

Paul-St-Young added 2 commits April 12, 2018 18:03

add estimator-sofk tests

67ac55c

add tests to check consistency between scalar.dat and stat.h5 output paths

add a symbolic link to bccH-1x1x1 wf h5

8948125

ghost assigned Paul-St-Young Apr 12, 2018

ghost added the in progress label Apr 12, 2018

Paul-St-Young requested review from markdewing and ye-luo April 12, 2018 23:23

Paul-St-Young added 3 commits April 13, 2018 14:37

pass argument to check script

c95318c

move math macros out of backflow section

9fac440

add estimator manager documentation

ec6778f

markdewing reviewed Apr 13, 2018

View reviewed changes

markdewing reviewed Apr 14, 2018

View reviewed changes

Merge remote-tracking branch 'github/develop' into dat-h5

3b3926b

ghost assigned ye-luo Nov 27, 2018

ye-luo approved these changes Dec 17, 2018

View reviewed changes

ye-luo changed the title ~~[WIP] make averages of Collectables and Properties consistent~~ Add sofk estimator test and manual entry about how estimator works Dec 17, 2018

ye-luo added 2 commits December 18, 2018 10:16

Merge branch 'develop' into dat-h5

fee7773

Merge branch 'develop' into dat-h5

8954f78

ye-luo merged commit ca320f4 into QMCPACK:develop Dec 18, 2018

ghost removed the in progress label Dec 18, 2018

prckent mentioned this pull request Dec 18, 2018

sofk estimator test uses python pandas #1273

Open

jtkrogel mentioned this pull request Jun 27, 2019

Tests wanted / List of untested functionality #1061

Open

37 tasks

tiihonej mentioned this pull request Aug 28, 2019

Fix AA fixed Coulomb energy observable #1841

Merged

Paul-St-Young deleted the dat-h5 branch October 26, 2019 16:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add sofk estimator test and manual entry about how estimator works #764

Add sofk estimator test and manual entry about how estimator works #764

Paul-St-Young commented Apr 12, 2018 •

edited by ye-luo

Loading

qmc-robot commented Apr 12, 2018

qmc-robot commented Apr 12, 2018

Paul-St-Young commented Apr 12, 2018

Paul-St-Young commented Apr 12, 2018

markdewing commented Apr 13, 2018

markdewing commented Apr 13, 2018

Paul-St-Young commented Apr 13, 2018

Paul-St-Young commented Apr 13, 2018

markdewing Apr 13, 2018

ye-luo Apr 13, 2018

Paul-St-Young Apr 14, 2018

markdewing Apr 13, 2018

Paul-St-Young Apr 14, 2018

markdewing Apr 13, 2018

jtkrogel Apr 16, 2018

markdewing Apr 13, 2018

markdewing Apr 14, 2018

markdewing Apr 14, 2018

prckent commented Nov 27, 2018

ye-luo commented Nov 27, 2018

ye-luo left a comment

qmc-robot commented Dec 17, 2018


		Two ensemble-level data structures \verb\|ParticleSet::Properties\| and \verb\|::Collectables\| serve as intermediaries between evaluate classes and output classes to scalar.dat and stat.h5. \verb\|Properties\| appears in both scalar.dat and stat.h5, whereas \verb\|Collectables\| appears only in stat.h5. \verb\|Properties\| is overwritten by \verb\|QMCHamiltonian::Observables\| at the end of each step. \verb\|QMCHamiltonian::Observables\| is filled upon call to \verb\|QMCHamiltonian::evaluate\| and \verb\|::auxHevaluate\|. \verb\|Collectables\| is zeroed at the beginning of each step and accumulated upon call to \verb\|::auxHevaluate\|.

		Data are outputted to scalar.dat in 4 stages: evaluate, load, unload, and collect. In the evaluate stage, \verb\|QMCHamiltonian::Observables\| is populated by a list of \verb\|QMCHamiltonianBase\|. In the load stage, \verb\|QMCHamiltonian::Observables\| is transfered to \verb\|Properties\| by \verb\|QMCDriver\|. In the unload stage, \verb\|Properties\| is copied to \verb\|LocalEnergyEstimator::scalars\|. In the collect stage, \verb\|LocalEnergyEstimator::scalars\| is block-averaged to \verb\|EstimatorManagerBase\|\\ \verb\|::AverageCache\| and dumped to file. For \verb\|Collectables\|, the evaluate and load stages are combined in a call to \verb\|QMCHamiltonian::auxHevaluate\|. In the unload stage, \verb\|Collectables\| is copied to \verb\|CollectablesEstimator::scalars\|. In the collect stage, \verb\|CollectablesEstimator\|\\ \verb\|::scalars\| is block-averaged to \verb\|EstimatorManagerBase::AverageCache\| and dumped to file.

Add sofk estimator test and manual entry about how estimator works #764

Add sofk estimator test and manual entry about how estimator works #764

Conversation

Paul-St-Young commented Apr 12, 2018 • edited by ye-luo Loading

qmc-robot commented Apr 12, 2018

qmc-robot commented Apr 12, 2018

Paul-St-Young commented Apr 12, 2018

Paul-St-Young commented Apr 12, 2018

markdewing commented Apr 13, 2018

markdewing commented Apr 13, 2018

Paul-St-Young commented Apr 13, 2018

Paul-St-Young commented Apr 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prckent commented Nov 27, 2018

ye-luo commented Nov 27, 2018

ye-luo left a comment

Choose a reason for hiding this comment

qmc-robot commented Dec 17, 2018

Paul-St-Young commented Apr 12, 2018 •

edited by ye-luo

Loading