SolutionArray/HDF interface for 1D FlameBase

[ischoegl]

Checklist

• There is a clear use-case for this code change
• The commit message has a short title & references relevant issues
• Build passes (scons build & scons test) and unit tests address code coverage
• The pull request is ready for review

If applicable, fill in the issue number this pull request is fixing

Partially addresses “big” project ideas (see 1D Flame Development Ideas):

Introduce HDF5 or CSV or JSON or YAML or some other output format for 1D flames to replace XML

Add ability to directly restore 1D flame objects from a saved solution (need to embed info about flame configuration and mechanism).

Changes proposed in this pull request

• Add SolutionArray input/output for 1D FlameBase objects (i.e. FreeFlame, etc.)
• HDF5 input/output
• Small fixes of SolutionArray glitches (i.e. slicing, extra column import)
• Restart via set_initial_guess

E.g. change to cantera/examples/onedim folder, and run (within ipython):

In [1]: %run adiabatic_flame.py
[...]
multicomponent flamespeed = 0.727008 m/s
Solution saved to file h2_adiabatic.xml as solution multi.
Solution saved to 'h2_adiabatic.csv'.

In [2]: arr = f.to_solution_array()
  ... : arr
Out[3]: <cantera.composite.SolutionArray at 0x7f9464f827f0>

In [3]: f1 = ct.FreeFlame(gas)
  ... : f1.from_solution_array(arr)

In [4]: all(f.grid == f1.grid)
Out[4]: True

In [5]: all(f.T == f1.T)
Out[5]: True

In [6]: f.to_pandas()
Out[6]: 
      grid  velocity            T   density      X_H2           X_H       X_O      X_O2      X_OH         X_H2O         X_HO2        X_H2O2      X_AR
0     -inf  0.727008   300.000000  1.338612  0.154930  0.000000e+00  0.000000  0.140845  0.000000  0.000000e+00  0.000000e+00  0.000000e+00  0.704225
1    0.000  0.727008   300.000000  1.338612  0.154930  1.407594e-16  0.000000  0.140845  0.000000  0.000000e+00  0.000000e+00  8.663079e-17  0.704225
2    0.006  0.727008   300.000000  1.338612  0.154930  0.000000e+00  0.000000  0.140845  0.000000  0.000000e+00  0.000000e+00  8.422789e-17  0.704225
3    0.012  0.727008   300.000000  1.338612  0.154930  0.000000e+00  0.000000  0.140845  0.000000  0.000000e+00  0.000000e+00  8.866101e-17  0.704225
4    0.015  0.727008   300.000000  1.338612  0.154930  0.000000e+00  0.000000  0.140845  0.000000  1.130882e-16  0.000000e+00  9.622272e-17  0.704225
..     ...       ...          ...       ...       ...           ...       ...       ...       ...           ...           ...           ...       ...
136  0.033  4.148451  1858.587531  0.234589  0.000072  6.253460e-06  0.000071  0.070944  0.001071  1.631748e-01  9.233693e-07  5.341914e-08  0.764660
137  0.036  4.149534  1859.115819  0.234528  0.000064  5.258929e-06  0.000063  0.070960  0.001012  1.632172e-01  8.979551e-07  5.130584e-08  0.764678
138  0.042  4.150244  1859.462329  0.234488  0.000059  4.657906e-06  0.000058  0.070971  0.000972  1.632451e-01  8.831509e-07  4.998669e-08  0.764689
139  0.048  4.150480  1859.577354  0.234474  0.000057  4.467670e-06  0.000057  0.070974  0.000959  1.632544e-01  8.785943e-07  4.956310e-08  0.764693
140  0.060  4.150480  1859.577354  0.234474  0.000057  4.467670e-06  0.000057  0.070974  0.000959  1.632544e-01  8.785943e-07  4.956310e-08  0.764693

[141 rows x 13 columns]

In [7]: f.write_hdf('solution.h5')

In [8]: f2 = ct.FreeFlame(gas)
   ...: f2.set_initial_guess(data='solution.h5')

In [9]: all(f.grid == f2.grid)
Out[9]: True

[ischoegl]

@speth (and @bryanwweber)... I am stopping here for feedback before adding a couple more tweaks (e.g. adding HDF-based set_initial_guess to other FlameBase objects).

[speth]

This is certainly interesting, and a good use of the work that you put into the HDF5 / pandas interface to the SolutionArray class. For the Python interface, this seems like it may be the best way to provide a more useful alternative to the XML output format. Ultimately, we still need some output format that can be used directly from C++, but this might save us from having to do anything annoying like adding a dependency on the HDF5 library.

I did have a few questions / comments:

• I'm not sure I understand the need to extend the interface of FreeFlame.set_initial_guess. Why is this different from just using from_solution_array?
• I think it's counterintuitive that from_solution_array only sets data for the 1D domain but does not set any of the boundary values. I'm not sure what the best way to handle this is. While in many cases, these values are the same as the first/last grid points, this is not always true (e.g. in the case of the mass fractions in BurnerFlame).
• Is there any way to extend this to include so that the saved data is as complete as what gets written to the XML output files? Some of these properties are per grid point, so the mapping is straightforward, but there are others that don't, e.g. the grid refinement tolerances and the species_enabled flag.

[ischoegl]

Is there any way to extend this to include so that the saved data is as complete as what gets written to the XML output files? Some of these properties are per grid point, so the mapping is straightforward, but there are others that don't, e.g. the grid refinement tolerances and the species_enabled flag.

Absolutely - at least for HDF (and minus the mechanism which imho should be dealt with separately). I mainly wanted to get feedback before addressing minor issues that will take some time and thought to implement.

[ischoegl]

@speth ... I addressed most of your comments (except for nomenclature I believe all are resolved).

Per your questions:

I'm not sure I understand the need to extend the interface of FreeFlame.set_initial_guess. Why is this different from just using from_solution_array?

The difference is motivated by restart applications which (1) avoid poor convergence for marginal conditions (i.e. the usual initial guess performs poorly), or (2) quickly fill flame data bases. Based on past experience, it is much quicker to assign a pruned solution from a related condition as an initial guess. Obviously, some small adjustments are needed (e.g. initial temperature), which is where differences arise.

I think it's counterintuitive that from_solution_array only sets data for the 1D domain but does not set any of the boundary values. I'm not sure what the best way to handle this is. While in many cases, these values are the same as the first/last grid points, this is not always true (e.g. in the case of the mass fractions in BurnerFlame).

This is an excellent point. My motivation was mainly restart via set_initial_guess, but I agree that everything should be restored as anticipated for from_solution_array as well. I believe this is best handled by a flag where the default behavior restores boundary conditions, while the set_initial_guess route leaves them alone.

[codecov]

Codecov Report

Merging #805 into master will increase coverage by 0.05%.
The diff coverage is 86.48%.

@@            Coverage Diff             @@
##           master     #805      +/-   ##
==========================================
+ Coverage   71.39%   71.45%   +0.05%     
==========================================
  Files         372      372              
  Lines       43482    43522      +40     
==========================================
+ Hits        31045    31098      +53     
+ Misses      12437    12424      -13     

Impacted Files	Coverage Δ
include/cantera/oneD/Domain1D.h	69.76% <ø> (ø)
include/cantera/oneD/Sim1D.h	62.50% <ø> (ø)
include/cantera/oneD/StFlow.h	91.83% <57.14%> (+1.13%)	⬆️
src/oneD/StFlow.cpp	89.52% <83.33%> (+2.58%)	⬆️
src/oneD/Sim1D.cpp	73.01% <100.00%> (+1.28%)	⬆️
src/oneD/Domain1D.cpp	58.46% <0.00%> (-3.85%)	⬇️
test/kinetics/kineticsFromYaml.cpp	97.75% <0.00%> (+0.10%)	⬆️
src/kinetics/KineticsFactory.cpp	96.34% <0.00%> (+1.34%)	⬆️
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e23dec...7b161e9. Read the comment docs.

[ischoegl]

@speth ... quick question regarding storage of simulation settings: I went through parameters that are accessible from Python, and came up with the following list:

['transport_model',
 'energy_enabled',
 'soret_enabled',
 'radiation_enabled',
 'ratio',
 'slope',
 'curve',
 'prune',
 'max_time_step_count',
 'max_grid_points',
 'steady_abstol',
 'steady_reltol',
 'transient_abstol',
 'transient_reltol']

where for tolerances it is straight-forward to store mode and deviating values to avoid clutter. It is straight-forward to save this in tabular form to the same HDF container that also holds the data.

Things that I anticipate to be missing at the moment (a few require the implementation of property getters):

• z_fixed / t_fixed
• boundary emissivities
• radiation qdot if radiation_enabled
• species_enabled ... missing in Python - is this relevant for FlameBase simulations?
• time stamp.

[ischoegl]

Here is a brief summary about settings output to HDF:

In [1]: %run adiabatic_flame.py  
[...]
multicomponent flamespeed = 0.727008 m/s
Solution saved to file h2_adiabatic.xml as solution multi.
Solution saved to 'h2_adiabatic.csv'.

In [2]: f.write_hdf('solution.h5')

In [3]: import pandas as pd

In [4]: with pd.HDFStore('solution.h5') as hdf:
    ...:     keys = hdf.keys()
    ...:     

In [5]: keys
Out[5]: ['/data', '/settings']

In [6]: s = pd.read_hdf('solution.h5', key='settings')

In [7]: s
Out[7]: 
    key                             date configuration transport_model  ...  steady_abstol  steady_reltol  transient_abstol  transient_reltol
0  data  Thu, 27 Feb 2020 21:00:23 +0000     FreeFlame           Multi  ...   1.000000e-09         0.0001      1.000000e-11            0.0001

[1 rows x 20 columns]

In [8]: list(s.columns)
Out[8]: 
['key',
 'date',
 'configuration',
 'transport_model',
 'energy_enabled',
 'soret_enabled',
 'radiation_enabled',
 'emissivity_left',
 'emissivity_right',
 'fixed_temperature',
 'ratio',
 'slope',
 'curve',
 'prune',
 'max_time_step_count',
 'max_grid_points',
 'steady_abstol',
 'steady_reltol',
 'transient_abstol',
 'transient_reltol']

[ischoegl]

@speth ... I will stop at this point for further feedback. At this point, I believe all that is left is:

1. settle on a replacement for the component name V (I am advocating for deprecation rather than local renaming - see velocity replacing u). Edit: I am proposing vGradient... This should eventually resolve the pre-existing band-aids
   

   cantera/interfaces/cython/cantera/onedim.py

   Lines 341 to 342 in c16c4b0

   	FlameBase.volume = _array_property('v') # avoid confusion with velocity gradient 'V'
   	FlameBase.int_energy = _array_property('u') # avoid collision with velocity 'u'

2. follow up on discussion of restart via set_initial_guess

PS: Not sure what's up with the code coverage (again) ... the issue is not with files that I touched ...

[ischoegl]

@speth ... your comment:

Is there any way to extend this to include so that the saved data is as complete as what gets written to the XML output files? Some of these properties are per grid point, so the mapping is straightforward, but there are others that don't, e.g. the grid refinement tolerances and the species_enabled flag.

prompted me to add this feature to this PR (this includes restoring of settings from HDF data). Regarding the species_enabled flag, note that the interface handles FlameBase, which is a subset of Domain1D where the flag is not being used. It's certainly possible to extend this idea to more generic cases, but I'd prefer to leave this for another PR.

I'd appreciate if you could have a look before this gets larger. Most of what I can think of is implemented at this point; what I anticipate beyond is mostly implementing set_initial_guess for FlameBase classes other than FreeFlame, and some tinkering with examples.

PS: I will not add anything beyond this point until I hear back.

PPS: Regarding the Boundary1D.h diff that doesn't show up in the overview, please refer to 6ea933f (which renames Brdy1D to Boundary1D, etc.), ec63aa2 (handling of deprecation) and f61592f (which replaces doublereal by double; since I'm touching the entire file, I felt like this was an appropriate opportunity).

[ischoegl]

Update: While I'm not adding anything, I moved unrelated renaming changes (Inlet1D.h / Bdry1D) to a separate PR #815 to avoid clutter.

[ischoegl]

Rebased as newly merged #816 pre-empted fixes within this PR.

PS: one potential addition would be to replace u and V by velocity and spread_rate in various CSV output scenarios for consistency. I am somewhat reluctant to touch this without prior discussion though.

PPS: the existing FlameBase.write_csv routines could be re-routed through SolutionArray.write_csv (see last commit). The advantage is to have a uniform interface that allows for re-importing of (reformatted) CSV output, a (minor) disadvantage is that units are lost from the column headers. For older Python versions, the order of onedim-specific columns may not be preserved, see #832.