Introduce Policy Methods for MOEA/D-DE

HISTORY.md

@@ -27,13 +27,16 @@
 
  * Make Callback flexible for MultiObjective Optimizers
    ([#289](https://github.com/mlpack/ensmallen/pull/289)).
-   
+
  * Add ZDT Test Suite
    ([#273](https://github.com/mlpack/ensmallen/pull/273)).
 
  * Add MOEA-D/DE Optimizer
    ([#269](https://github.com/mlpack/ensmallen/pull/269)).
 
+ * Introduce Policy Methods for MOEA/D-DE
+   ([#293](https://github.com/mlpack/ensmallen/pull/293)).
+
 ### ensmallen 2.16.1: "Severely Dented Can Of Polyurethane"
 ###### 2021-03-02
  * Fix test compilation issue when `ENS_USE_OPENMP` is set

doc/optimizers.md

@@ -1573,8 +1573,29 @@ decomposed to form a Single Objective Problem. A diversity preserving mechanism
 a varied set of solution.
 
 #### Constructors
-* `MOEAD()`
-* `MOEAD(`_`populationSize, maxGenerations, crossoverProb,  neighborProb, neighborSize, distributionIndex, differentialWeight, maxReplace, epsilon, lowerBound, upperBound`_`)`
+* `MOEAD<`_`InitPolicyType, DecompPolicyType`_`>()`
+* `MOEAD<`_`InitPolicyType, DecompPolicyType`_`>(`_`populationSize, maxGenerations, crossoverProb,  neighborProb, neighborSize, distributionIndex, differentialWeight, maxReplace, epsilon, lowerBound, upperBound`_`)`
+
+The _`InitPolicyType`_ template parameter refers to the strategy used to
+initialize the reference directions.
+
+The following types are available:
+
+ * **`BayesianBootstrap`**
+
+The _`DecompPolicyType`_ template parameter refers to the strategy used to
+decompose the weight vectors to form a scalar objective function.
+
+The following types are available:
+
+ * **`Tchebycheff`**
+ * **`WeightedAverage`**
+ * **`PenaltyBoundaryIntersection`**
+
+For convenience the following types can be used:
+
+ * **`DefaultMOEAD`** (equivalent to `MOEAD<BayesianBootstrap, Tchebycheff>`): utilizes BayesianBootstrap for weight init
+ and Tchebycheff for weight decomposition.
 
 #### Attributes
 
@@ -1591,10 +1612,12 @@ a varied set of solution.
 | `double` | **`epsilon`** | Handles numerical stability after weight initialization. | `1E-10`|
 | `double`, `arma::vec` | **`lowerBound`** | Lower bound of the coordinates on the coordinates of the whole population during the search process. | `0` |
 | `double`, `arma::vec` | **`upperBound`** | Lower bound of the coordinates on the coordinates of the whole population during the search process. | `1` |
+| `InitPolicyType` | **`initPolicy`** | Instantiated init policy used to initialize weights. | `InitPolicyType()` |
+| `DecompPolicyType` | **`decompPolicy`** | Instantiated decomposition policy used to create scalar objective problem. | `DecompPolicyType()` |
 
 Attributes of the optimizer may also be changed via the member methods
 `PopulationSize()`, `MaxGenerations()`, `CrossoverRate()`, `NeighborProb()`, `NeighborSize()`, `DistributionIndex()`,
-`DifferentialWeight()`, `MaxReplace()`, `Epsilon()`, `LowerBound()` and `UpperBound()`.
+`DifferentialWeight()`, `MaxReplace()`, `Epsilon()`, `LowerBound()`, `UpperBound()`, `InitPolicy()` and `DecompPolicy()`.
 
 #### Examples:
 
@@ -1606,7 +1629,7 @@ Attributes of the optimizer may also be changed via the member methods
 SchafferFunctionN1<arma::mat> SCH;
 arma::vec lowerBound("-10 -10");
 arma::vec upperBound("10 10");
-MOEAD opt(150, 300, 1.0, 0.9, 20, 20, 0.5, 2, 1E-10, lowerBound, upperBound);
+DefaultMOEAD opt(150, 300, 1.0, 0.9, 20, 20, 0.5, 2, 1E-10, lowerBound, upperBound);
 typedef decltype(SCH.objectiveA) ObjectiveTypeA;
 typedef decltype(SCH.objectiveB) ObjectiveTypeB;
 arma::mat coords = SCH.GetInitialPoint();

include/ensmallen_bits/moead/decomposition_policies/pbi_decomposition.hpp

@@ -0,0 +1,81 @@
+/**
+ * @file pbi_decomposition.hpp
+ * @author Nanubala Gnana Sai
+ *
+ * The Penalty Based Boundary Intersection (PBI) decomposition policy.
+ *
+ * ensmallen is free software; you may redistribute it and/or modify it under
+ * the terms of the 3-clause BSD license.  You should have received a copy of
+ * the 3-clause BSD license along with ensmallen.  If not, see
+ * http://www.opensource.org/licenses/BSD-3-Clause for more information.
+ */
+#ifndef ENSMALLEN_MOEAD_PBI_HPP
+#define ENSMALLEN_MOEAD_PBI_HPP
+
+namespace ens {
+
+/**
+ * Penalty Based Boundary Intersection (PBI) method is a weight decomposition method,
+ * it tries to find the intersection between bottom-most boundary of the attainable
+ * objective set with the reference directions.
+ *
+ * The goal is to minimize the distance between objective vectors with the ideal point
+ * along the reference direction. To handle equality constraints, a penalty parameter
+ * theta is used.
+ *
+ * For more information, see the following:
+ * @code
+ * article{zhang2007moea,
+ *   title={MOEA/D: A multiobjective evolutionary algorithm based on decomposition},
+ *   author={Zhang, Qingfu and Li, Hui},
+ *   journal={IEEE Transactions on evolutionary computation},
+ *   pages={712--731},
+ *   year={2007}
+ * @endcode
+ */
+class PenaltyBoundaryIntersection
+{
+ public:
+  /**
+   * Constructor for Penalty Based Boundary Intersection decomposition
+   * policy.
+   *
+   * @param theta The penalty value.
+   */
+  PenaltyBoundaryIntersection(const double theta = 5) :
+      theta(theta)
+  {
+    /* Nothing to do. */
+  }
+
+  /**
+   * Decompose the weight vectors.
+   *
+   * @tparam VecType The type of the vector used in the decommposition.
+   * @param weight The weight vector corresponding to a subproblem.
+   * @param idealPoint The reference point in the objective space.
+   * @param candidateFitness The objective vector of the candidate.
+   */
+  template<typename VecType>
+  typename VecType::elem_type Apply(const VecType& weight,
+                                    const VecType& idealPoint,
+                                    const VecType& candidateFitness)
+  {
+    typedef typename VecType::elem_type ElemType;
+    //! A unit vector in the same direction as the provided weight vector.
+    const VecType referenceDirection = weight / arma::norm(weight, 1);
+    //! Distance of F(x) from the idealPoint along the reference direction.
+    const ElemType d1 = arma::dot(candidateFitness - idealPoint, referenceDirection);
+    //! The perpendicular distance of F(x) from reference direction.
+    const ElemType d2 = arma::norm(candidateFitness - (idealPoint + d1 * referenceDirection), 1);
+
+    return d1 + static_cast<ElemType>(theta) * d2;
+  }
+
+  private:
+    double theta;
+};
+
+} // namespace ens
+
+#endif

include/ensmallen_bits/moead/decomposition_policies/tchebycheff_decomposition.hpp

@@ -0,0 +1,66 @@
+/**
+ * @file tchebycheff_decomposition.hpp
+ * @author Nanubala Gnana Sai
+ *
+ * The Tchebycheff Weight decomposition policy.
+ *
+ * ensmallen is free software; you may redistribute it and/or modify it under
+ * the terms of the 3-clause BSD license.  You should have received a copy of
+ * the 3-clause BSD license along with ensmallen.  If not, see
+ * http://www.opensource.org/licenses/BSD-3-Clause for more information.
+ */
+#ifndef ENSMALLEN_MOEAD_TCHEBYCHEFF_HPP
+#define ENSMALLEN_MOEAD_TCHEBYCHEFF_HPP
+
+namespace ens {
+
+/**
+ * The Tchebycheff method works by taking the maximum of element-wise product
+ * between reference direction and the line connecting objective vector and
+ * ideal point.
+ *
+ * Under mild conditions, for each Pareto Optimal point there exists a reference
+ * direction such that the given point is also the optimal solution
+ * to this scalar objective.
+ *
+ * For more information, see the following:
+ * @code
+ * article{zhang2007moea,
+ *   title={MOEA/D: A multiobjective evolutionary algorithm based on decomposition},
+ *   author={Zhang, Qingfu and Li, Hui},
+ *   journal={IEEE Transactions on evolutionary computation},
+ *   pages={712--731},
+ *   year={2007}
+ * @endcode
+ */
+class Tchebycheff
+{
+ public:
+  /**
+   * Constructor for Tchebycheff decomposition policy.
+   */
+  Tchebycheff()
+  {
+    /* Nothing to do. */
+  }
+
+  /**
+   * Decompose the weight vectors.
+   *
+   * @tparam VecType The type of the vector used in the decommposition.
+   * @param weight The weight vector corresponding to a subproblem.
+   * @param idealPoint The reference point in the objective space.
+   * @param candidateFitness The objective vector of the candidate.
+   */
+  template<typename VecType>
+  typename VecType::elem_type Apply(const VecType& weight,
+                                    const VecType& idealPoint,
+                                    const VecType& candidateFitness)
+  {
+    return arma::max(weight % arma::abs(candidateFitness - idealPoint));
+  }
+};
+
+} // namespace ens
+
+#endif

include/ensmallen_bits/moead/decomposition_policies/weighted_decomposition.hpp

@@ -0,0 +1,62 @@
+/**
+ * @file weighted_decomposition.hpp
+ * @author Nanubala Gnana Sai
+ *
+ * The Weighted Average decomposition policy.
+ *
+ * ensmallen is free software; you may redistribute it and/or modify it under
+ * the terms of the 3-clause BSD license.  You should have received a copy of
+ * the 3-clause BSD license along with ensmallen.  If not, see
+ * http://www.opensource.org/licenses/BSD-3-Clause for more information.
+ */
+#ifndef ENSMALLEN_MOEAD_WEIGHTED_HPP
+#define ENSMALLEN_MOEAD_WEIGHTED_HPP
+
+namespace ens {
+
+/**
+ * The Weighted average method of decomposition. The working principle is to
+ * minimize the dot product between reference direction and the line connecting
+ * objective vector and ideal point.
+ *
+ * For more information, see the following:
+ * @code
+ * article{zhang2007moea,
+ *   title={MOEA/D: A multiobjective evolutionary algorithm based on decomposition},
+ *   author={Zhang, Qingfu and Li, Hui},
+ *   journal={IEEE Transactions on evolutionary computation},
+ *   pages={712--731},
+ *   year={2007}
+ * @endcode
+ */
+class WeightedAverage
+{
+ public:
+  /**
+   * Constructor for Weighted Average decomposition policy.
+   */
+  WeightedAverage()
+  {
+    /* Nothing to do. */
+  }
+
+  /**
+   * Decompose the weight vectors.
+   *
+   * @tparam VecType The type of the vector used in the decommposition.
+   * @param weight The weight vector corresponding to a subproblem.
+   * @param idealPoint The reference point in the objective space.
+   * @param candidateFitness The objective vector of the candidate.
+   */
+  template<typename VecType>
+  typename VecType::elem_type Apply(const VecType& weight,
+                                    const VecType& /* idealPoint */,
+                                    const VecType& candidateFitness)
+  {
+    return arma::dot(weight, candidateFitness);
+  }
+};
+
+} // namespace ens
+
+#endif