Skip to content

Commit

Permalink
LAMMPSBaseParser: Fix the parsing of the performance data
Browse files Browse the repository at this point in the history
The parser attempts to extract the measure of the steps per second from
the logfile through a regex to match the following syntax:

    Performance: 39.271 ns/day, 0.611 hours/ns, 45452.422 timesteps/s

However, this line can also appear in the form:

    Performance: 140354.357 tau/day, 324.894 timesteps/s, 162.447 Matom-step/s

The regex is updated to be indifferent as to the position of the measure
and simply look for the float preceding the `timesteps/s` marker. A test
is added with an example log file with the latter format.
  • Loading branch information
sphuber committed Mar 23, 2023
1 parent 3167f7f commit 48c224b
Show file tree
Hide file tree
Showing 4 changed files with 115 additions and 6 deletions.
11 changes: 5 additions & 6 deletions aiida_lammps/common/raw_parsers.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,13 @@ def parse_logfile(filename: str = None, file_contents: str = None) -> Union[dict
end_found = False
parsed_data = {}
global_parsed_data = {}
perf_regex = re.compile(
r"Performance\:\s(.+)\sns\/day,\s(.+)\shours\/ns\,\s(.+)\stimesteps\/s\s*"
)

perf_regex = re.compile(r".*Performance\:.*\,\s+([0-9\.]*)\stimesteps\/s.*")
performance_match = perf_regex.search(file_contents or "")
if performance_match:
global_parsed_data["steps_per_second"] = float(performance_match.group(1))

for index, line in enumerate(data):
if perf_regex.match(line):
_, _, step_sec = perf_regex.match(line).groups()
global_parsed_data["steps_per_second"] = float(step_sec)
if "binsize" in line:
global_parsed_data["binsize"] = ast.literal_eval(
line.split()[2].replace(",", "")
Expand Down
60 changes: 60 additions & 0 deletions tests/input_files/parsers/log_alt.lammps
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
LAMMPS (3 Nov 2022)
In general, for best performance with OpenMP 4.0 or better set OMP_PROC_BIND=spread and OMP_PLACES=threads
For best performance with OpenMP 3.1 set OMP_PROC_BIND=true
For unit testing set OMP_PROC_BIND=false
using 1 OpenMP thread(s) per MPI task
Lattice spacing in x,y,z = 1.6795962 1.6795962 1.6795962
Created orthogonal box = (0 0 0) to (83.97981 83.97981 83.97981)
6 by 4 by 5 MPI processor grid
Created 500000 atoms
using lattice units in orthogonal box = (0 0 0) to (83.97981 83.97981 83.97981)
create_atoms CPU = 0.003 seconds
Generated 0 of 0 mixed pair_coeff terms from geometric mixing rule
Neighbor list info ...
update: every = 20 steps, delay = 0 steps, check = no
max neighbors/atom: 2000, page size: 100000
master list distance cutoff = 2.8
ghost atom cutoff = 2.8
binsize = 1.4, bins = 60 60 60
1 neighbor lists, perpetual/occasional/extra = 1 0 0
(1) pair lj/cut/kk, perpetual
attributes: half, newton on, kokkos_device
pair build: half/bin/kk/device
stencil: half/bin/3d
bin: kk/device
Setting up Verlet run ...
Unit style : lj
Current step : 0
Time step : 0.005
Per MPI rank memory allocation (min/avg/max) = 2.039 | 2.065 | 2.082 Mbytes
Step Temp E_pair E_mol TotEng Press
0 1.44 -6.7733681 0 -4.6133724 -5.0196717
1000 0.70380068 -5.6761498 0 -4.6204508 0.70370101
Loop time of 3.07792 on 120 procs for 1000 steps with 500000 atoms

Performance: 140354.357 tau/day, 324.894 timesteps/s, 162.447 Matom-step/s
99.4% CPU use with 120 MPI tasks x 1 OpenMP threads

MPI task timing breakdown:
Section | min time | avg time | max time |%varavg| %total
---------------------------------------------------------------
Pair | 1.6398 | 1.7065 | 1.7679 | 2.2 | 55.44
Neigh | 0.33289 | 0.34561 | 0.39959 | 1.4 | 11.23
Comm | 0.8538 | 0.95433 | 1.0342 | 3.8 | 31.01
Output | 0.00017888 | 0.00025242 | 0.0003793 | 0.0 | 0.01
Modify | 0.053668 | 0.055804 | 0.058023 | 0.4 | 1.81
Other | | 0.01547 | | | 0.50

Nlocal: 4166.67 ave 4239 max 4109 min
Histogram: 5 4 16 24 34 11 14 8 1 3
Nghost: 5717.37 ave 5790 max 5627 min
Histogram: 3 2 8 10 21 20 31 11 10 4
Neighs: 156192 ave 160654 max 152682 min
Histogram: 4 6 13 25 27 23 12 7 2 1

Total # of neighbors = 18743047
Ave neighs/atom = 37.486094
Neighbor list builds = 50
Dangerous builds not checked
final_etotal: -4.62045083768868
Total wall time: 0:00:03
35 changes: 35 additions & 0 deletions tests/test_parsers.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,41 @@ def test_lammps_base(db_test_app, data_regression):
data_regression.check({"results": calcfunction.outputs.results.get_dict()})


def test_lammps_base_timing_info(db_test_app, data_regression):
"""Test that the timing performance is properly parsed."""
filename = os.path.join(
TEST_DIR,
"input_files",
"parsers",
"log_alt.lammps",
)
retrieved = FolderData()
retrieved.base.repository.put_object_from_file(filename, "log.lammps")
retrieved.base.repository.put_object_from_filelike(
io.StringIO(""), "_scheduler-stdout.txt"
)
retrieved.base.repository.put_object_from_filelike(
io.StringIO(""), "_scheduler-stderr.txt"
)
inputs = {"script": SinglefileData(io.StringIO(""))}
calc_node = db_test_app.generate_calcjob_node(
"lammps.base", retrieved, inputs=inputs
)
parser = ParserFactory("lammps.base")
with db_test_app.sandbox_folder() as temp_path:
(
results,
calcfunction,
) = parser.parse_from_node( # pylint: disable=unused-variable
calc_node,
retrieved_temporary_folder=temp_path.abspath,
)

assert calcfunction.is_finished_ok
assert "results" in calcfunction.outputs
data_regression.check({"results": calcfunction.outputs.results.get_dict()})


@pytest.mark.parametrize(
"plugin_name", ["lammps.force", "lammps.optimize", "lammps.md", "lammps.md.multi"]
)
Expand Down
15 changes: 15 additions & 0 deletions tests/test_parsers/test_lammps_base_timing_info.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
results:
compute_variables:
bin: kk/device
bins:
- 60
- 60
- 60
binsize: 1.4
ghost_atom_cutoff: 2.8
master_list_distance_cutoff: 2.8
max_neighbors_atom: 2000
steps_per_second: 324.894
total_wall_time: 0:00:03
total_wall_time_seconds: 3
units_style: lattice

0 comments on commit 48c224b

Please sign in to comment.