merging

Docs/source/developers/checksum.rst

@@ -32,7 +32,7 @@ You can execute ``checksumAPI.py`` as a Python script for that, and pass the plo
 
 .. code-block:: bash
 
-   ./checksumAPI.py --evaluate --plotfile <path/to/plotfile> --test-name <test name>
+   ./checksumAPI.py --evaluate --output-file <path/to/plotfile> --format <'openpmd' or 'plotfile'> --test-name <test name>
 
 See additional options
 
@@ -41,17 +41,17 @@ See additional options
 * ``--rtol`` relative tolerance for the comparison
 * ``--atol`` absolute tolerance for the comparison (a sum of both is used by ``numpy.isclose()``)
 
-Reset a benchmark with new values that you know are correct
------------------------------------------------------------
+Create/Reset a benchmark with new values that you know are correct
+------------------------------------------------------------------
 
-Reset a benchmark from a plotfile generated locally
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Create/Reset a benchmark from a plotfile generated locally
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 This is using ``checksumAPI.py`` as a Python script.
 
 .. code-block:: bash
 
-   ./checksumAPI.py --reset-benchmark --plotfile <path/to/plotfile> --test-name <test name>
+   ./checksumAPI.py --reset-benchmark --output-file <path/to/plotfile> --format <'openpmd' or 'plotfile'> --test-name <test name>
 
 See additional options
 

Docs/source/usage/parameters.rst

@@ -81,6 +81,43 @@ Overall simulation parameters
     one should not expect to obtain the same random numbers,
     even if a fixed ``warpx.random_seed`` is provided.
 
+* ``algo.evolve_scheme`` (`string`, default: `explicit`)
+    Specifies the evolve scheme used by WarpX.
+
+    * ``explicit``: Use an explicit solver, such as the standard FDTD or PSATD
+
+    * ``implicit_picard``: Use an implicit solver with exact energy conservation that uses a Picard iteration to solve the system.
+      Note that this method is for demonstration only. It is inefficient and does not work well when
+      :math:`\omega_{pe} \Delta t` is close to or greater than one.
+      The method is described in `Angus et al., On numerical energy conservation for an implicit particle-in-cell method coupled with a binary Monte-Carlo algorithm for Coulomb collisions <https://doi.org/10.1016/j.jcp.2022.111030>`__.
+      The version implemented is an updated version that is relativistically correct, including the relativistic gamma factor for the particles.
+      For exact energy conservation, ``algo.current_deposition = direct`` must be used with ``interpolation.galerkin_scheme = 0``,
+      and ``algo.current_deposition = Esirkepov`` must be used with ``interpolation.galerkin_scheme = 1`` (which is the default, in
+      which case charge will also be conserved).
+
+    * ``semi_implicit_picard``: Use an energy conserving semi-implicit solver that uses a Picard iteration to solve the system.
+      Note that this method has the CFL limitation :math:`\Delta t < c/\sqrt( \sum_i 1/\Delta x_i^2 )`. It is inefficient and does not work well or at all when :math:`\omega_{pe} \Delta t` is close to or greater than one.
+      The method is described in `Chen et al., A semi-implicit, energy- and charge-conserving particle-in-cell algorithm for the relativistic Vlasov-Maxwell equations <https://doi.org/10.1016/j.jcp.2020.109228>`__.
+      For energy conservation, ``algo.current_deposition = direct`` must be used with ``interpolation.galerkin_scheme = 0``,
+      and ``algo.current_deposition = Esirkepov`` must be used with ``interpolation.galerkin_scheme = 1`` (which is the default, in
+      which case charge will also be conserved).
+
+* ``algo.max_picard_iterations`` (`integer`, default: 10)
+    When `algo.evolve_scheme` is either `implicit_picard` or `semi_implicit_picard`, this sets the maximum number of Picard
+    itearations that are done each time step.
+
+* ``algo.picard_iteration_tolerance`` (`float`, default: 1.e-7)
+    When `algo.evolve_scheme` is either `implicit_picard` or `semi_implicit_picard`, this sets the convergence tolerance of
+    the iterations, the maximum of the relative change of the L2 norm of the field from one iteration to the next.
+    If this is set to zero, the maximum number of iterations will always be done with the change only calculated on the last
+    iteration (for a slight optimization).
+
+* ``algo.require_picard_convergence`` (`bool`, default: 1)
+    When `algo.evolve_scheme` is either `implicit_picard` or `semi_implicit_picard`, this sets whether the iteration each step
+    is required to converge.
+    If it is required, an abort is raised if it does not converge and the code then exits.
+    If not, then a warning is issued and the calculation continues.
+
 * ``warpx.do_electrostatic`` (`string`) optional (default `none`)
     Specifies the electrostatic mode. When turned on, instead of updating
     the fields at each iteration with the full Maxwell equations, the fields

Examples/Physics_applications/laser_acceleration/README.rst

@@ -5,8 +5,9 @@ Laser-Wakefield Acceleration of Electrons
 
 This example shows how to model a laser-wakefield accelerator (LWFA) :cite:p:`ex-TajimaDawson1982,ex-Esarey1996`.
 
-Laser-wakefield acceleration is best performed in 3D and quasi-cylindrical (RZ) geometry, which ensures that the plasma wavelength of the wakefield is modelled with the right scale lengths.
-RZ modeling enables efficient modeling if effects of asymmetry shall be ignored (e.g., asymmetric beams and transverse profiles, hosing of the injected beam, etc.).
+Laser-wakefield acceleration is best performed in 3D or quasi-cylindrical (RZ) geometry, in order to correctly capture some of the key physics (laser diffraction, beamloading, shape of the accelerating bubble in the blowout regime, etc.).
+For physical situations that have close-to-cylindrical symmetry, simulations in RZ geometry capture the relevant physics at a fraction of the computational cost of a 3D simulation.
+On the other hand, for physical situation with strong asymmetries (e.g., non-round laser driver, strong hosing of the accelerated beam, etc.), only 3D simulations are suitable.
 
 For LWFA scenarios with long propagation lengths, use the :ref:`boosted frame method <theory-boostedframe>`.
 An example can be seen in the :ref:`PWFA example <examples-pwfa>`.

Examples/Physics_applications/plasma_acceleration/README.rst

@@ -5,8 +5,9 @@ Beam-Driven Wakefield Acceleration of Electrons
 
 This example shows how to model a beam-driven plasma-wakefield accelerator (PWFA) :cite:p:`ex-TajimaDawson1982,ex-Esarey1996`.
 
-PWFA is best performed in 3D and quasi-cylindrical (RZ) geometry, which ensures that the plasma wavelength of the wakefield is modelled with the right scale lengths.
-RZ modeling enables efficient modeling if effects of asymmetry shall be ignored (e.g., asymmetric beams and transverse profiles, hosing of the injected beam, etc.).
+PWFA is best performed in 3D or quasi-cylindrical (RZ) geometry, in order to correctly capture some of the key physics (structure of the space-charge fields, beamloading, shape of the accelerating bubble in the blowout regime, etc.).
+For physical situations that have close-to-cylindrical symmetry, simulations in RZ geometry capture the relevant physics at a fraction of the computational cost of a 3D simulation.
+On the other hand, for physical situation with strong asymmetries (e.g., non-round driver, strong hosing of the accelerated beam, etc.), only 3D simulations are suitable.
 
 Additionally, to speed up computation, this example uses the :ref:`boosted frame method <theory-boostedframe>` to effectively model long acceleration lengths.
 

Examples/Tests/Implicit/analysis_1d.py

@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+
+# Copyright 2023 David Grote
+#
+#
+# This file is part of WarpX.
+#
+# License: BSD-3-Clause-LBNL
+#
+# This is a script that analyses the simulation results from
+# the script `inputs_1d`. This simulates a 1D periodic plasma using the implicit solver.
+import os
+import sys
+
+import numpy as np
+
+sys.path.insert(1, '../../../../warpx/Regression/Checksum/')
+import checksumAPI
+
+# this will be the name of the plot file
+fn = sys.argv[1]
+
+field_energy = np.loadtxt('diags/reducedfiles/field_energy.txt', skiprows=1)
+particle_energy = np.loadtxt('diags/reducedfiles/particle_energy.txt', skiprows=1)
+
+total_energy = field_energy[:,2] + particle_energy[:,2]
+
+delta_E = (total_energy - total_energy[0])/total_energy[0]
+max_delta_E = np.abs(delta_E).max()
+
+tolerance_rel = 1.e-14
+
+print(f"max change in energy: {max_delta_E}")
+print(f"tolerance: {tolerance_rel}")
+
+assert( max_delta_E < tolerance_rel )
+
+test_name = os.path.split(os.getcwd())[1]
+checksumAPI.evaluate_checksum(test_name, fn)

Examples/Tests/Implicit/inputs_1d

@@ -0,0 +1,81 @@
+#################################
+############ CONSTANTS #############
+#################################
+
+my_constants.n0 = 1.e30          # plasma densirty, m^-3
+my_constants.nz = 40             # number of grid cells
+my_constants.Ti = 100.           # ion temperature, eV
+my_constants.Te = 100.           # electron temperature, eV
+my_constants.wpe = q_e*sqrt(n0/(m_e*epsilon0))  # electron plasma frequency, radians/s
+my_constants.de0 = clight/wpe    # skin depth, m
+my_constants.nppcz = 100         # number of particles/cell in z
+my_constants.dt = 0.1/wpe        # time step size, s
+
+#################################
+####### GENERAL PARAMETERS ######
+#################################
+
+max_step = 100
+amr.n_cell = nz
+amr.max_level = 0
+
+geometry.dims = 1
+geometry.prob_lo = 0.0
+geometry.prob_hi = 10.*de0
+boundary.field_lo = periodic
+boundary.field_hi = periodic
+boundary.particle_lo = periodic
+boundary.particle_hi = periodic
+
+#################################
+############ NUMERICS ###########
+#################################
+
+warpx.const_dt = dt
+algo.evolve_scheme = implicit_picard
+algo.max_picard_iterations = 31
+algo.picard_iteration_tolerance = 0.
+algo.current_deposition = esirkepov
+algo.field_gathering = energy-conserving
+algo.particle_shape = 2
+warpx.use_filter = 0
+
+#################################
+############ PLASMA #############
+#################################
+
+particles.species_names = electrons protons
+
+electrons.species_type = electron
+electrons.injection_style = "NUniformPerCell"
+electrons.num_particles_per_cell_each_dim = nppcz
+electrons.profile = constant
+electrons.density = n0
+electrons.momentum_distribution_type = gaussian
+electrons.ux_th = sqrt(Te*q_e/m_e)/clight
+electrons.uy_th = sqrt(Te*q_e/m_e)/clight
+electrons.uz_th = sqrt(Te*q_e/m_e)/clight
+
+protons.species_type = proton
+protons.injection_style = "NUniformPerCell"
+protons.num_particles_per_cell_each_dim = nppcz
+protons.profile = constant
+protons.density = n0
+protons.momentum_distribution_type = gaussian
+protons.ux_th = sqrt(Ti*q_e/m_p)/clight
+protons.uy_th = sqrt(Ti*q_e/m_p)/clight
+protons.uz_th = sqrt(Ti*q_e/m_p)/clight
+
+# Diagnostics
+diagnostics.diags_names = diag1
+diag1.intervals = 100
+diag1.diag_type = Full
+diag1.fields_to_plot = Ex Ey Ez Bx By Bz jx jy jz rho divE
+diag1.electrons.variables = w ux uy uz
+diag1.protons.variables = w ux uy uz
+
+warpx.reduced_diags_names = particle_energy field_energy
+particle_energy.type = ParticleEnergy
+particle_energy.intervals = 1
+field_energy.type = FieldEnergy
+field_energy.intervals = 1

Examples/Tests/ionization/analysis_ionization.py

@@ -93,5 +93,13 @@
 
 assert( error_rel < tolerance_rel )
 
+# Check that the user runtime component (if it exists) worked as expected
+try:
+    orig_z = ad['electrons', 'particle_orig_z'].v
+    assert np.all( (orig_z > 0) & (orig_z < 1.5e-5) )
+    print('particle_orig_z has reasonable values')
+except yt.utilities.exceptions.YTFieldNotFound:
+    pass # Some of the tested script to not have the quantity orig_z
+
 test_name = os.path.split(os.getcwd())[1]
 checksumAPI.evaluate_checksum(test_name, filename)

Examples/Tests/ionization/inputs_2d_rt

@@ -36,6 +36,8 @@ ions.physical_element = N
 electrons.mass = m_e
 electrons.charge = -q_e
 electrons.injection_style = none
+electrons.addRealAttributes = orig_z
+electrons.attribute.orig_z(x,y,z,ux,uy,uz,t) = z
 
 lasers.names        = laser1
 laser1.profile      = Gaussian

Regression/Checksum/benchmarks_json/ImplicitPicard_1d.json

@@ -0,0 +1,29 @@
+{
+  "lev=0": {
+    "Bx": 3730.0029376363264,
+    "By": 1593.5906541698305,
+    "Bz": 0.0,
+    "Ex": 797541065253.7858,
+    "Ey": 981292393404.0359,
+    "Ez": 3528134993266.091,
+    "divE": 2.0829069134855788e+21,
+    "jx": 7.639291641209293e+17,
+    "jy": 1.4113587963237038e+18,
+    "jz": 1.3506587033985085e+18,
+    "rho": 18442449008.581665
+  },
+  "protons": {
+    "particle_momentum_x": 5.231105747759245e-19,
+    "particle_momentum_y": 5.367982834807453e-19,
+    "particle_momentum_z": 5.253213507906386e-19,
+    "particle_position_x": 0.00010628272743703996,
+    "particle_weight": 5.314093261582036e+22
+  },
+  "electrons": {
+    "particle_momentum_x": 1.196379551301037e-20,
+    "particle_momentum_y": 1.2271443795645239e-20,
+    "particle_momentum_z": 1.2277752539495415e-20,
+    "particle_position_x": 0.00010649569055433632,
+    "particle_weight": 5.314093261582036e+22
+  }
+}

Regression/Checksum/benchmarks_json/ionization_lab.json

@@ -3,6 +3,7 @@
     "particle_momentum_x": 4.407898469197755e-18,
     "particle_momentum_y": 0.0,
     "particle_momentum_z": 2.642991174223682e-18,
+    "particle_orig_z": 0.43016526372226926,
     "particle_position_x": 0.1095015206652257,
     "particle_position_y": 0.6413864600981052,
     "particle_weight": 3.443203125e-10

Regression/WarpX-tests.ini

@@ -4462,3 +4462,20 @@ compileTest = 0
 doVis = 0
 compareParticles = 0
 analysisRoutine = Examples/Tests/nodal_electrostatic/analysis_3d.py
+
+[ImplicitPicard_1d]
+buildDir = .
+inputFile = Examples/Tests/Implicit/inputs_1d
+runtime_params = warpx.abort_on_warning_threshold=high
+dim = 1
+addToCompileString =
+cmakeSetupOpts = -DWarpX_DIMS=1
+restartTest = 0
+useMPI = 1
+numprocs = 2
+useOMP = 0
+numthreads = 1
+compileTest = 0
+doVis = 0
+compareParticles = 1
+analysisRoutine = Examples/Tests/Implicit/analysis_1d.py

Source/Diagnostics/WarpXOpenPMD.cpp

@@ -877,7 +877,11 @@ WarpXOpenPMDPlot::SaveRealProperty (ParticleIter& pti,
 
 {
   auto const numParticleOnTile = pti.numParticles();
+<<<<<<< HEAD
   uint64_t const numParticleOnTile64 = static_cast<uint64_t>( numParticleOnTile );
+=======
+  auto const numParticleOnTile64 = static_cast<uint64_t>(numParticleOnTile);
+>>>>>>> 514df482be5a2565be56acf5fc4b9bcc9d89273b
   auto const& soa = pti.GetStructOfArrays();
 
   auto const getComponentRecord = [&currSpecies](std::string const comp_name) {

Source/Evolve/CMakeLists.txt

@@ -4,5 +4,6 @@ foreach(D IN LISTS WarpX_DIMS)
       PRIVATE
         WarpXEvolve.cpp
         WarpXComputeDt.cpp
+        WarpXOneStepImplicitPicard.cpp
     )
 endforeach()

Source/Evolve/Make.package

@@ -1,4 +1,5 @@
 CEXE_sources += WarpXEvolve.cpp
 CEXE_sources += WarpXComputeDt.cpp
+CEXE_sources += WarpXOneStepImplicitPicard.cpp
 
 VPATH_LOCATIONS   += $(WARPX_HOME)/Source/Evolve