Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable serialization of Cantera objects #984

Merged
merged 57 commits into from
Apr 14, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
a20e9ae
[Input] Add a basic YAML emitter for AnyMap / AnyValue
speth Nov 3, 2019
a74e763
[Input] Implement serialization of species thermo objects
speth Nov 12, 2019
d59142f
[Input] Make simple ThermoPhase objects serializable
speth Nov 12, 2019
f0cf42e
[Input] Make Species objects serializable, including transport data
speth Nov 16, 2019
26562ae
[Input] Introduce YamlWriter class for input file serialization
speth Nov 16, 2019
8ec3fed
[Input] Implement serialization of gas phase reactions
speth Nov 17, 2019
ec83f30
[Input] Implement serialization of interface reactions
speth Nov 24, 2019
af0576d
[Input] Add reactions/kinetics to YamlWriter output
speth Nov 25, 2019
51a32b4
[Input] Add transport model specification to YamlWriter output
speth Nov 25, 2019
c09f7e3
[Input] Implement serialization of more complex phase types
speth Jan 30, 2021
65dbcd3
[Input] Improve float formatting in YAML output
speth Dec 13, 2019
07e8ef5
[Input] Use stream-based YAML output method to allow formatting control
speth Dec 16, 2019
2d9bd81
[Input] Enable optional use of flow mode for outputting YAML maps
speth Jan 5, 2020
c5c8bdf
[Input] Preserve order of items when outputting YAML maps
speth Jan 5, 2020
19f4ed9
[Input] Retain input species thermo data
speth Jan 7, 2020
57d4392
[Input] Add option to skip user-defined fields when writing to YAML
speth Jan 7, 2020
8ac4271
[Input] Add pathway for serialization of equation-of-state data
speth Jan 7, 2020
af89ec5
[Input] Serialize equation-of-state data for IdealSolidSolnPhase
speth Jan 7, 2020
a6ad50b
[Input] Serialize equation-of-state data for LatticePhase/LatticeSoli…
speth Jan 7, 2020
5cd2750
[Input] Serialize equation-of-state data for StoichSubstance
speth Jan 7, 2020
747624e
[Input] Enable serialization of Redlich-Kwong species
speth Jan 7, 2020
9e7e9c4
[Input] Serialize PDSS_IonsFromNeutral
speth Jan 7, 2020
7c77d32
[Input] Serialize PDSS_ConstVol, PDSS_IdealGas, PDSS_Water
speth Jan 7, 2020
ee9d9eb
[Input] Serialize PDSS_HKFT
speth Jan 12, 2020
af08cd8
[Input] Implement serialization of PDSS_SSVol
speth Jan 12, 2020
472e89f
[Input] Serialize species-specific data for DebyeHuckel model
speth Jan 13, 2020
ad3cd4f
[Input] YamlWriter can use separate Thermo/Kinetics/Transport objects
speth Jan 13, 2020
8b15f4d
[Input] Serialize phase state using native state variables
speth Jan 13, 2020
e314cfe
[Input] Fix serialization when species has no thermo model
speth Jan 13, 2020
1059fe3
[Test] Add round-trip serialization tests for thermo models
speth Jan 13, 2020
c023796
[Python] Add conversion of AnyMap to native Python types
speth Jan 15, 2020
ae8d0b5
[Python] Add access to input data for Solution objects
speth Jan 21, 2020
55c6360
[Python] Add access to input data for Species objects
speth Jan 21, 2020
5b00695
Add extra information to AnyValue::getMapWhere error messages
speth Jan 22, 2020
c117ece
[Python] Add access to input data for Reaction objects
speth Jan 22, 2020
1f6e690
Add functions for conversion from given units into a UnitSystem
speth Jan 22, 2020
0e85f20
Make AnyMap::applyUnits idempotent
speth Jan 23, 2020
697d45c
[Input] Provide control over ordering of YAML fields
speth Feb 3, 2021
b92d6ae
[Input] Enable conversion of added data to units of the parent AnyMap
speth Mar 6, 2021
cdc7241
[Input] Share UnitSystems among nested AnyMaps
speth Mar 6, 2021
51ad58e
[Input] Add control over units of constructed AnyMap objects
speth Mar 8, 2021
2cd7e9a
[Input] Add option of setting units to YamlWriter
speth Mar 8, 2021
6d60e9c
[Input] Handle variable output units for all reaction types
speth Mar 9, 2021
cbb8568
Eliminate redundancy in implementation of AnyValue::as
speth Mar 10, 2021
f20c774
[Input] Handle variable output units for all thermo/species thermo types
speth Mar 10, 2021
aac2c5a
[Input] Add tests of YamlWriter for surface and edge phases
speth Mar 10, 2021
4df6886
[Input/Python] Enable generation of YAML files from Python Solutions
speth Mar 10, 2021
85fb401
[Input] Serialize Redlich-Kwong binary interaction parameters
speth Mar 13, 2021
22f703b
[Input] Set rate_units for reactions whenever possible
speth Apr 10, 2021
bc61d76
[Input] Fix key ordering when outputting to Python
speth Apr 11, 2021
6609866
[Input] Use full AnyMap sorting algorithm in all cases
speth Apr 11, 2021
8574c1c
[Input] Refactor to reduce Python manipulations of AnyMap
speth Apr 12, 2021
ece7c0d
[Python/Input] Make function names more Pythonic
speth Apr 12, 2021
daf1f42
[Input/Examples] Update Python examples to demonstrate YAML output
speth Apr 12, 2021
2e0061e
[Input/Test] Add tests of solution properties for round-tripped phases
speth Apr 12, 2021
96808a9
[Input] Fix AnyMap handling of 2D string / boolean arrays
speth Apr 13, 2021
c8aea00
[Input/Test] Improve test coverage of YAML serialization
speth Apr 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 170 additions & 11 deletions include/cantera/base/AnyMap.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ namespace boost
class any;
}

namespace YAML
{
class Emitter;
Emitter& operator<<(Emitter& out, const Cantera::AnyMap& rhs);
Emitter& operator<<(Emitter& out, const Cantera::AnyValue& rhs);
}

namespace Cantera
{

Expand All @@ -38,10 +45,12 @@ class AnyBase {
const AnyValue& getMetadata(const std::string& key) const;

protected:
//! Line where this node occurs in the input file
//! The line where this value occurs in the input file. Set to -1 for values
//! that weren't created from an input file.
int m_line;

//! Column where this node occurs in the input file
//! If m_line >= 0, the column where this value occurs in the input file.
//! If m_line == -1, a value used for determining output ordering
int m_column;

//! Metadata relevant to an entire AnyMap tree, such as information about
Expand Down Expand Up @@ -129,6 +138,33 @@ class AnyValue : public AnyBase
friend bool operator==(const std::string& lhs, const AnyValue& rhs);
friend bool operator!=(const std::string& lhs, const AnyValue& rhs);

//! @name Quantity conversions
//! Assign a quantity consisting of one or more values and their
//! corresponding units, which will be converted to a target unit system
//! when the applyUnits() function is later called on the root of the
//! AnyMap.
//! @{

//! Assign a scalar quantity with units as a string, for example
//! `{3.0, "m^2"}`. If the `is_act_energy` flag is set to `true`, the units
//! will be converted using the special rules for activation energies.
void setQuantity(double value, const std::string& units, bool is_act_energy=false);

//! Assign a scalar quantity with units as a Units object, for cases where
//! the units vary and are determined dynamically, such as reaction
//! pre-exponential factors
void setQuantity(double value, const Units& units);

//! Assign a vector where all the values have the same units
void setQuantity(const vector_fp& values, const std::string& units);

typedef std::function<void(AnyValue&, const UnitSystem&)> unitConverter;

//! Assign a value of any type where the unit conversion requires a
//! different behavior besides scaling all values by the same factor
void setQuantity(const AnyValue& value, const unitConverter& converter);
//! @} end group quantity conversions

explicit AnyValue(double value);
AnyValue& operator=(double value);
//! Return the held value as a `double`, if it is a `double` or a `long
Expand Down Expand Up @@ -217,8 +253,14 @@ class AnyValue : public AnyBase
//! Returns `true` when getMapWhere() would succeed
bool hasMapWhere(const std::string& key, const std::string& value) const;

//! @see AnyMap::applyUnits
void applyUnits(const UnitSystem& units);
//! Return values used to determine the sort order when outputting to YAML
std::pair <int, int> order() const;

//! @see AnyMap::applyUnits(const UnitSystem&)
void applyUnits(shared_ptr<UnitSystem>& units);

//! @see AnyMap::setFlowStyle
void setFlowStyle(bool flow=true);

private:
std::string demangle(const std::type_info& type) const;
Expand Down Expand Up @@ -254,6 +296,8 @@ class AnyValue : public AnyBase
static bool vector2_eq(const boost::any& lhs, const boost::any& rhs);

mutable Comparer m_equals;

friend YAML::Emitter& YAML::operator<<(YAML::Emitter& out, const AnyValue& rhs);
};

//! Implicit conversion to vector<AnyValue>
Expand Down Expand Up @@ -355,7 +399,7 @@ std::vector<AnyMap>& AnyValue::asVector<AnyMap>(size_t nMin, size_t nMax);
class AnyMap : public AnyBase
{
public:
AnyMap(): m_units() {};
AnyMap();

//! Create an AnyMap from a YAML file.
/*!
Expand All @@ -369,10 +413,17 @@ class AnyMap : public AnyBase
//! Create an AnyMap from a string containing a YAML document
static AnyMap fromYamlString(const std::string& yaml);

std::string toYamlString() const;

//! Get the value of the item stored in `key`.
AnyValue& operator[](const std::string& key);
const AnyValue& operator[](const std::string& key) const;

//! Used to create a new item which will be populated from a YAML input
//! string, where the item with `key` occurs at the specified line and
//! column within the string.
AnyValue& createForYaml(const std::string& key, int line, int column);

//! Get the value of the item stored in `key`. Raises an exception if the
//! value does not exist.
const AnyValue& at(const std::string& key) const;
Expand All @@ -386,6 +437,10 @@ class AnyMap : public AnyBase
//! Erase all items in the mapping
void clear();

//! Add items from `other` to this AnyMap. If keys in `other` also exist in
//! this AnyMap, the `keepExisting` option determines which item is used.
void update(const AnyMap& other, bool keepExisting=true);

//! Return a string listing the keys in this AnyMap, e.g. for use in error
//! messages
std::string keys_str() const;
Expand Down Expand Up @@ -447,6 +502,7 @@ class AnyMap : public AnyBase
//! skips over keys that start and end with double underscores.
class Iterator {
public:
Iterator() {}
Iterator(const std::unordered_map<std::string, AnyValue>::const_iterator& start,
const std::unordered_map<std::string, AnyValue>::const_iterator& stop);

Expand Down Expand Up @@ -476,6 +532,55 @@ class AnyMap : public AnyBase
return Iterator(m_data.end(), m_data.end());
}

class OrderedIterator;

//! Proxy for iterating over an AnyMap in the defined output ordering.
//! See ordered().
class OrderedProxy {
public:
OrderedProxy() {}
OrderedProxy(const AnyMap& data);
OrderedIterator begin() const;
OrderedIterator end() const;

typedef std::vector<std::pair<
std::pair<int, int>,
const std::pair<const std::string, AnyValue>*>> OrderVector;
private:
const AnyMap* m_data;
OrderVector m_ordered;
std::unique_ptr<std::pair<const std::string, AnyValue>> m_units;
};

//! Defined to allow the OrderedProxy class to be used with range-based
//! for loops.
class OrderedIterator {
public:
OrderedIterator() {}
OrderedIterator(const OrderedProxy::OrderVector::const_iterator& start,
const OrderedProxy::OrderVector::const_iterator& stop);

const std::pair<const std::string, AnyValue>& operator*() const {
return *m_iter->second;
}
const std::pair<const std::string, AnyValue>* operator->() const {
return &(*m_iter->second);
}
bool operator!=(const OrderedIterator& right) const {
return m_iter != right.m_iter;
}
OrderedIterator& operator++() { ++m_iter; return *this; }

private:
OrderedProxy::OrderVector::const_iterator m_iter;
OrderedProxy::OrderVector::const_iterator m_stop;
};

// Return a proxy object that allows iteration in an order determined by the
// order of insertion, the location in an input file, and rules specified by
// the addOrderingRules() method.
OrderedProxy ordered() const { return OrderedProxy(*this); }

//! Returns the number of elements in this map
size_t size() const {
return m_data.size();
Expand All @@ -485,7 +590,7 @@ class AnyMap : public AnyBase
bool operator!=(const AnyMap& other) const;

//! Return the default units that should be used to convert stored values
const UnitSystem& units() const { return m_units; }
const UnitSystem& units() const { return *m_units; }

//! Use the supplied UnitSystem to set the default units, and recursively
//! process overrides from nodes named `units`.
Expand All @@ -496,28 +601,82 @@ class AnyMap : public AnyBase
* then the specified units are taken to be the defaults for all the maps in
* the list.
*
* After being processed, the `units` nodes are removed, so this function
* should be called only once, on the root AnyMap. This function is called
* automatically by the fromYamlFile() and fromYamlString() constructors.
* After being processed, the `units` nodes are removed. This function is
* called automatically by the fromYamlFile() and fromYamlString()
* constructors.
*
* @warning This function is an experimental part of the %Cantera API and
* may be changed or removed without notice.
*/
void applyUnits(const UnitSystem& units);
void applyUnits();

//! @see applyUnits(const UnitSystem&)
void applyUnits(shared_ptr<UnitSystem>& units);

//! Set the unit system for this AnyMap. The applyUnits() method should be
//! called on the root AnyMap object after all desired calls to setUnits()
//! in the tree have been made.
void setUnits(const UnitSystem& units);

//! Use "flow" style when outputting this AnyMap to YAML
void setFlowStyle(bool flow=true);

//! Add global rules for setting the order of elements when outputting
//! AnyMap objects to YAML
/*!
* Enables specifying keys that should appear at either the beginning
* or end of the generated YAML mapping. Only programmatically-added keys
* are rearranged. Keys which come from YAML input retain their existing
* ordering, and are output after programmatically-added keys.
*
* This function should be called exactly once for any given spec that
* is to be added. To facilitate this, the method returns a bool so that
* it can be called as part of initializing a static variable. To avoid
* spurious compiler warnings about unused variables, the following
* structure can be used:
*
* ```
* static bool reg = AnyMap::addOrderingRules("Reaction",
* {{"head", "equation"}, {"tail", "duplicate"}});
* if (reg) {
* reactionMap["__type__"] = "Reaction";
* }
* ```
*
* @param objectType Apply rules to maps where the hidden `__type__` key
* has the corresponding value.
* @param specs A list of rule specifications. Each rule consists of
* two strings. The first string is either "head" or "tail", and the
* second string is the name of a key
* @returns ``true``, to facilitate static initialization
*/
static bool addOrderingRules(const std::string& objectType,
const std::vector<std::vector<std::string>>& specs);

private:
//! The stored data
std::unordered_map<std::string, AnyValue> m_data;

//! The default units that are used to convert stored values
UnitSystem m_units;
std::shared_ptr<UnitSystem> m_units;

//! Cache for previously-parsed input (YAML) files. The key is the full path
//! to the file, and the second element of the value is the last-modified
//! time for the file, which is used to enable change detection.
static std::unordered_map<std::string, std::pair<AnyMap, int>> s_cache;

//! Information about fields that should appear first when outputting to
//! YAML. Keys in this map are matched to `__type__` keys in AnyMap
//! objects, and values are a list of field names.
static std::unordered_map<std::string, std::vector<std::string>> s_headFields;

//! Information about fields that should appear last when outputting to
//! YAML. Keys in this map are matched to `__type__` keys in AnyMap
//! objects, and values are a list of field names.
static std::unordered_map<std::string, std::vector<std::string>> s_tailFields;

friend class AnyValue;
friend YAML::Emitter& YAML::operator<<(YAML::Emitter& out, const AnyMap& rhs);
};

// Define begin() and end() to allow use with range-based for loops
Expand Down
33 changes: 15 additions & 18 deletions include/cantera/base/AnyMap.inl.h
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ const T &AnyValue::as() const {
// Implicit conversion of long int to double
*m_value = static_cast<double>(as<long int>());
m_equals = eq_comparer<double>;
} else if (typeid(T) == typeid(std::vector<double>)
&& m_value->type() == typeid(std::vector<AnyValue>)) {
// Implicit conversion of vector<AnyValue> to vector<double>
auto& asAny = as<std::vector<AnyValue>>();
vector_fp asDouble(asAny.size());
for (size_t i = 0; i < asAny.size(); i++) {
asDouble[i] = asAny[i].as<double>();
}
*m_value = std::move(asDouble);
m_equals = eq_comparer<std::vector<double>>;
}
return boost::any_cast<const T&>(*m_value);
} catch (boost::bad_any_cast&) {
Expand All @@ -37,31 +47,18 @@ const T &AnyValue::as() const {

template<class T>
T &AnyValue::as() {
try {
if (typeid(T) == typeid(double) && m_value->type() == typeid(long int)) {
// Implicit conversion of long int to double
*m_value = static_cast<double>(as<long int>());
m_equals = eq_comparer<double>;
}
return boost::any_cast<T&>(*m_value);
} catch (boost::bad_any_cast&) {
if (m_value->type() == typeid(void)) {
// Values that have not been set are of type 'void'
throw InputFileError("AnyValue::as", *this,
"Key '{}' not found or contains no value", m_key);
} else {
throw InputFileError("AnyValue::as", *this,
"Key '{}' contains a '{}',\nnot a '{}'",
m_key, demangle(m_value->type()), demangle(typeid(T)));
}
}
// To avoid duplicating the code from the const version, call that version
// and just remove the const specifier from the return value
return const_cast<T&>(const_cast<const AnyValue*>(this)->as<T>());
}

template<class T>
bool AnyValue::is() const {
return m_value->type() == typeid(T);
}

template<> bool AnyValue::is<std::vector<double>>() const;

template<class T>
AnyValue &AnyValue::operator=(const std::vector<T> &value) {
*m_value = value;
Expand Down
3 changes: 3 additions & 0 deletions include/cantera/base/Solution.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ namespace Cantera
class ThermoPhase;
class Kinetics;
class Transport;
class AnyMap;

//! A container class holding managers for all pieces defining a phase
class Solution : public std::enable_shared_from_this<Solution>
Expand Down Expand Up @@ -61,6 +62,8 @@ class Solution : public std::enable_shared_from_this<Solution>
return m_transport;
}

AnyMap parameters(bool withInput=false) const;

protected:
shared_ptr<ThermoPhase> m_thermo; //!< ThermoPhase manager
shared_ptr<Kinetics> m_kinetics; //!< Kinetics manager
Expand Down
Loading