Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for CMA-ES inconsistencies. #193

Closed
wants to merge 22 commits into from
Closed

Fix for CMA-ES inconsistencies. #193

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
f428f71
Initial commit.
gaurav-singh1998 Apr 4, 2020
c0a5f28
Changes in cmaes test.
gaurav-singh1998 Apr 4, 2020
1e848fd
Update include/ensmallen_bits/cmaes/cmaes_impl.hpp
gaurav-singh1998 Apr 8, 2020
2e503f1
Update include/ensmallen_bits/cmaes/cmaes_impl.hpp
gaurav-singh1998 Apr 8, 2020
93146c5
Update include/ensmallen_bits/cmaes/cmaes_impl.hpp
gaurav-singh1998 Apr 8, 2020
1edbfe7
Update include/ensmallen_bits/cmaes/cmaes_impl.hpp
gaurav-singh1998 Apr 8, 2020
7de865b
Update include/ensmallen_bits/cmaes/cmaes_impl.hpp
gaurav-singh1998 Apr 8, 2020
659d0c6
Changes in implementation.
gaurav-singh1998 Apr 8, 2020
748bbd4
Minor style fixes.
gaurav-singh1998 Apr 8, 2020
9ad2852
Changes in inverse implementation.
gaurav-singh1998 Apr 8, 2020
8d7598c
Merge branch 'master' into cmaes-fix
gaurav-singh1998 May 9, 2020
63aed56
Changes in Documentation and some other minor changes.
gaurav-singh1998 May 9, 2020
77a89ca
Merge branch 'master' into cmaes-fix
gaurav-singh1998 May 9, 2020
881de8a
Changes in HISTORY.md
gaurav-singh1998 May 9, 2020
2ec0daa
Changing pow to sqrt.
gaurav-singh1998 May 14, 2020
c5bebd6
Changing ens::Warn to assert.
gaurav-singh1998 May 15, 2020
89fe19c
Update include/ensmallen_bits/cmaes/cmaes.hpp
gaurav-singh1998 May 15, 2020
2b86192
Adding boundary transformation test.
gaurav-singh1998 May 24, 2020
838e47c
Merge branch 'master' into cmaes-fix
zoq Jul 14, 2020
486930a
Changed the test function from SGD to Logistic Regression.
gaurav-singh1998 Oct 4, 2020
bd793ed
Merge branch 'cmaes-fix' of https://github.com/gaurav-singh1998/ensma…
gaurav-singh1998 Oct 4, 2020
04ab903
Merge branch 'master' into cmaes-fix
gaurav-singh1998 Oct 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion HISTORY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
* Fix CMake package export
([#198](https://github.com/mlpack/ensmallen/pull/198)).

* Fix CMA-ES inconsistencies
([#193](https://github.com/mlpack/ensmallen/pull/193)).

### ensmallen 2.12.1: "Stir Crazy"
###### 2020-04-20
* Fix total number of epochs and time estimation for ProgressBar callback
Expand All @@ -20,7 +23,7 @@
([#183](https://github.com/mlpack/ensmallen/pull/183)).

* Remove deprecated methods from PrimalDualSolver implementation
([#185](https://github.com/mlpack/ensmallen/pull/185).
([#185](https://github.com/mlpack/ensmallen/pull/185)).

* Update logo ([#186](https://github.com/mlpack/ensmallen/pull/186)).

Expand Down
4 changes: 3 additions & 1 deletion doc/optimizers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,7 @@ matrix within an iterative procedure using the covariance matrix.
* `CMAES<`_`SelectionPolicyType`_`>(`_`lambda, lowerBound, upperBound`_`)`
* `CMAES<`_`SelectionPolicyType`_`>(`_`lambda, lowerBound, upperBound, batchSize`_`)`
* `CMAES<`_`SelectionPolicyType`_`>(`_`lambda, lowerBound, upperBound, batchSize, maxIterations, tolerance, selectionPolicy`_`)`
* `CMAES<`_`SelectionPolicyType`_`>(`_`lambda, lowerBound, upperBound, batchSize, maxIterations, tolerance, selectionPolicy, initialSigma`_`)`

The _`SelectionPolicyType`_ template parameter refers to the strategy used to
compute the (approximate) objective function. The `FullSelection` and
Expand All @@ -606,10 +607,11 @@ For convenience the following types can be used:
| `size_t` | **`maxIterations`** | Maximum number of iterations. | `1000` |
| `double` | **`tolerance`** | Maximum absolute tolerance to terminate algorithm. | `1e-5` |
| `SelectionPolicyType` | **`selectionPolicy`** | Instantiated selection policy used to calculate the objective. | `SelectionPolicyType()` |
| `double` | **`initialSigma`** | The initial step size. | `0.6` |

Attributes of the optimizer may also be changed via the member methods
`Lambda()`, `LowerBound()`, `UpperBound()`, `BatchSize()`, `MaxIterations()`,
`Tolerance()`, and `SelectionPolicy()`.
`Tolerance()`, `SelectionPolicy()` and `InitialSigma()`.

The `selectionPolicy` attribute allows an instantiated `SelectionPolicyType` to
be given. The `FullSelection` policy has no need to be instantiated and thus
Expand Down
28 changes: 22 additions & 6 deletions include/ensmallen_bits/cmaes/cmaes.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ class CMAES
* @param maxIterations Maximum number of iterations allowed (0 means no
* limit).
* @param tolerance Maximum absolute tolerance to terminate algorithm.
* @param initialSigma The initial step size.
* @param selectionPolicy Instantiated selection policy used to calculate the
* objective.
*/
Expand All @@ -75,7 +76,8 @@ class CMAES
const size_t batchSize = 32,
const size_t maxIterations = 1000,
const double tolerance = 1e-5,
const SelectionPolicyType& selectionPolicy = SelectionPolicyType());
const SelectionPolicyType& selectionPolicy = SelectionPolicyType(),
const double initialSigma = 0.6);
gaurav-singh1998 marked this conversation as resolved.
Show resolved Hide resolved

/**
* Optimize the given function using CMA-ES. The given starting point will be
Expand All @@ -97,19 +99,19 @@ class CMAES
MatType& iterate,
CallbackTypes&&... callbacks);

//! Get the step size.
//! Get the population size.
gaurav-singh1998 marked this conversation as resolved.
Show resolved Hide resolved
size_t PopulationSize() const { return lambda; }
//! Modify the step size.
//! Modify the population size.
size_t& PopulationSize() { return lambda; }

//! Get the lower bound of decision variables.
double LowerBound() const { return lowerBound; }
//! Modify the lower bound of decision variables.
double& LowerBound() { return lowerBound; }

//! Get the upper bound of decision variables
//! Get the upper bound of decision variables.
double UpperBound() const { return upperBound; }
//! Modify the upper bound of decision variables
//! Modify the upper bound of decision variables.
double& UpperBound() { return upperBound; }

//! Get the batch size.
Expand All @@ -132,14 +134,19 @@ class CMAES
//! Modify the selection policy.
SelectionPolicyType& SelectionPolicy() { return selectionPolicy; }

//! Get the initial step size.
double InitialSigma() const { return initialSigma; }
gaurav-singh1998 marked this conversation as resolved.
Show resolved Hide resolved
//! Modify the initial step size.
double& InitialSigma() { return initialSigma; }

private:
//! Population size.
size_t lambda;

//! Lower bound of decision variables.
double lowerBound;

//! Upper bound of decision variables
//! Upper bound of decision variables.
double upperBound;

//! The batch size for processing.
Expand All @@ -153,6 +160,15 @@ class CMAES

//! The selection policy used to calculate the objective.
SelectionPolicyType selectionPolicy;

//! Initial step size.
double initialSigma;

//! Methods used to transform the candidates into the constraints.
template<typename BaseMatType>
void BoundaryTransform(BaseMatType& matrix);
template<typename BaseMatType>
void BoundaryTransformInverse(BaseMatType& matrix);
};

/**
Expand Down
107 changes: 101 additions & 6 deletions include/ensmallen_bits/cmaes/cmaes_impl.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

// In case it hasn't been included yet.
#include "cmaes.hpp"
#include <assert.h>

#include <ensmallen_bits/function.hpp>

Expand All @@ -29,15 +30,20 @@ CMAES<SelectionPolicyType>::CMAES(const size_t lambda,
const size_t batchSize,
const size_t maxIterations,
const double tolerance,
const SelectionPolicyType& selectionPolicy) :
const SelectionPolicyType& selectionPolicy,
const double initialSigma) :
lambda(lambda),
lowerBound(lowerBound),
upperBound(upperBound),
batchSize(batchSize),
maxIterations(maxIterations),
tolerance(tolerance),
selectionPolicy(selectionPolicy)
{ /* Nothing to do. */ }
selectionPolicy(selectionPolicy),
initialSigma(initialSigma)
{
assert(this->lowerBound != this->upperBound && "The values of "
"lowerbound and upperbound must be different.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either camelcase the variable names or say lower bound.

}

//! Optimize the function (minimize).
template<typename SelectionPolicyType>
Expand Down Expand Up @@ -78,7 +84,7 @@ typename MatType::elem_type CMAES<SelectionPolicyType>::Optimize(

// Step size control parameters.
BaseMatType sigma(2, 1); // sigma is vector-shaped.
sigma(0) = 0.3 * (upperBound - lowerBound);
sigma(0) = initialSigma;
const double cs = (muEffective + 2) / (iterate.n_elem + muEffective + 5);
const double ds = 1 + cs + 2 * std::max(std::sqrt((muEffective - 1) /
(iterate.n_elem + 1)) - 1, 0.0);
Expand All @@ -99,13 +105,16 @@ typename MatType::elem_type CMAES<SelectionPolicyType>::Optimize(

std::vector<BaseMatType> mPosition(2, BaseMatType(iterate.n_rows,
iterate.n_cols));
mPosition[0] = lowerBound + arma::randu<BaseMatType>(
iterate.n_rows, iterate.n_cols) * (upperBound - lowerBound);
BaseMatType initialVal;
initialVal.randu(iterate.n_rows, iterate.n_cols);
initialVal += (BaseMatType)(iterateIn);
mPosition[0] = initialVal;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's still valid to apply the lower/upper bound here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @zoq so should I remove the boundary transformation from here and implement it like it was before like here. Thanks


BaseMatType step(iterate.n_rows, iterate.n_cols);
step.zeros();

// Calculate the first objective function.
BoundaryTransform<BaseMatType>(mPosition[0]);
ElemType currentObjective = 0;
for (size_t f = 0; f < numFunctions; f += batchSize)
{
Expand Down Expand Up @@ -177,6 +186,7 @@ typename MatType::elem_type CMAES<SelectionPolicyType>::Optimize(
pPosition[idx(j)] = mPosition[idx0] + sigma(idx0) * pStep[idx(j)];

// Calculate the objective function.
BoundaryTransform<BaseMatType>(pPosition[idx(j)]);
pObjective(idx(j)) = selectionPolicy.Select(function, batchSize,
pPosition[idx(j)], callbacks...);
}
Expand All @@ -191,6 +201,7 @@ typename MatType::elem_type CMAES<SelectionPolicyType>::Optimize(
mPosition[idx1] = mPosition[idx0] + sigma(idx0) * step;

// Calculate the objective function.
BoundaryTransform<BaseMatType>(mPosition[idx1]);
currentObjective = selectionPolicy.Select(function, batchSize,
mPosition[idx1], callbacks...);

Expand Down Expand Up @@ -313,6 +324,90 @@ typename MatType::elem_type CMAES<SelectionPolicyType>::Optimize(
return overallObjective;
}

// Transforms the candidate into the given bounds.
template<typename SelectionPolicyType>
template<typename BaseMatType>
void CMAES<SelectionPolicyType>::BoundaryTransform(BaseMatType& matrix)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I dived into the code and I think it makes sense to introduce another policy e.g. TransformationPolicy so we can enable/disable the boundary constraint. Let me know what you think, I can push the changes to you branch, if you like?

Copy link
Contributor Author

@gaurav-singh1998 gaurav-singh1998 Feb 7, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @zoq thanks for the review. I will try and implement the TransformationPolicy this week. Can you point to some source code to which I can refer before implementing this? Thank you.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So basically it's similar to how the update policies are implemented https://github.com/mlpack/ensmallen/tree/master/include/ensmallen_bits/sgd/update_policies.

We basically add a new template type called TransformationPolicy or something like that and define an interface. I think we just need two functions at this point Transform and Inverse. So the class policy class for the BoundaryBoxConstraint could look like:

class BoundaryBoxConstraint
{
 public:
  void Transform(...)
  {
  }

  void Inverse(...)
  {
  }
}

we also a policy class that does nothing:

class EmptyTransformation
{
 public:
  void Transform(...)
  {
  }

  void Inverse(...)
  {
  }
}

in the CMAES class we can than use an instantiation of the TransformationPolicy and call the Transform and Inverse methods. Hopefully what I wrote above makes sense, happy to clarify anything.

{
typedef typename BaseMatType::elem_type ElemType;
const double diff = (upperBound - lowerBound) / 2.0;
const double al = std::min(diff, (1 + std::abs(lowerBound)) / 20.0);
const double au = std::min(diff, (1 + std::abs(upperBound)) / 20.0);
const double xlow = lowerBound - 2 * al - diff;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should make t possible to define bounds for all coordinates similar to what has been done for the PSO method:

ensmallen/include/ensmallen_bits/pso/pso.hpp

Lines 91 to 92 in 70a2be8

const arma::mat& lowerBound = arma::ones(1, 1),
const arma::mat& upperBound = arma::ones(1, 1),

It uses a 1x1 matrix as a default, so the current implementation but you use a matrix of the same size as the coordinates as well. Let me know what you think. We can also implement the change another time and merge this as is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zoq, I implemented this boundary handing function after referring the N Hansen's implementation and in my opinion, I think we should merge this as it is and if there are any issues in the future I will try to change this accordingly. Thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, fine for me.

const double xup = upperBound + 2 * au + diff;
const double r = 2 * (upperBound - lowerBound + al + au);

for (size_t col = 0; col < matrix.n_cols; col++)
{
for (size_t row = 0; row < matrix.n_rows; row++)
{
ElemType y = matrix(row, col);
// Boundary transformation shift into feasible pre-image.
if (y < xlow)
{
y += (ElemType)(r * (1 + (xlow - y) / r));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be y += (ElemType)(r * (1 + (int)(xlow - y) / r)); also is the cast to ElemType needed, not sure I can see the reason to do that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @zoq sorry for replying so late. I did it to ensure that every element of the matrix are of same types if not required I'll remove it. Thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't mind let's remove the extra cast.

}
else if (y > xup)
{
y -= (ElemType)(r * (1 + (y - xup) / r));
}
else if (y < lowerBound - al)
{
y += (ElemType)(2 * (lowerBound - al - y));
}
else if (y > upperBound + au)
{
y -= (ElemType)(2 * (y - upperBound - au));
}
// Boundary transformation.
if (y < lowerBound + al)
{
y = (ElemType)(lowerBound + (y - (lowerBound - al)) *
(y - (lowerBound - al)) / 4.0 / al);
}
else if (y > upperBound - au)
{
y = (ElemType)(upperBound - (y - (upperBound + au)) *
(y - (upperBound + au)) / 4.0 / au);
}

matrix(row, col) = y;
}
}
}

// Computes the inverse of the transformation.
template<typename SelectionPolicyType>
template<typename BaseMatType>
void CMAES<SelectionPolicyType>::BoundaryTransformInverse(BaseMatType& matrix)
{
typedef typename BaseMatType::elem_type ElemType;
const double diff = (upperBound - lowerBound) / 2.0;
const double al = std::min(diff, (1 + std::abs(lowerBound)) / 20.0);
const double au = std::min(diff, (1 + std::abs(upperBound)) / 20.0);

for (size_t col = 0; col < matrix.n_cols; col++)
{
for (size_t row = 0; row < matrix.n_rows; row++)
{
ElemType y = matrix(row, col);

if (y < lowerBound + al)
{
y = (ElemType)(lowerBound - al) + 2 *
std::sqrt(std::abs(al * (y - lowerBound)));
}
else if (y > upperBound - au)
{
y = (ElemType)(upperBound + au) - 2 *
std::sqrt(std::abs(au * (upperBound - y)));
}

matrix(row, col) = y;
}
}
}

} // namespace ens

#endif
8 changes: 4 additions & 4 deletions tests/cmaes_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ TEST_CASE("CMAESLogisticRegressionTest", "[CMAESTest]")
responses, testResponses, shuffledResponses);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a test that makes sure CMAES keeps the answers within the given bounds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @favre49 sorry for being late on this. Can you please elaborate on the test for this should be written? Should I consider an armadillo matrix which is out of bounds and then apply the transformation to see if the values are within the specified range? Sorry, that I didn't get the gist of what you were saying.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant we should add a test that ensures the optimized coordinates are within the given bounds [-20,20]. Something like REQUIRE(coordinates[i] <=20 && coordinates[i] >= -20)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @favre49 the test has been added kindly have a look whenever you get a chance. Thanks.

LogisticRegression<> lr(shuffledData, shuffledResponses, 0.5);

CMAES<> cmaes(0, -1, 1, 32, 200, 1e-3);
CMAES<> cmaes(0, -20, 20, 32, 200, 1e-3);
arma::mat coordinates = lr.GetInitialPoint();
cmaes.Optimize(lr, coordinates);

Expand Down Expand Up @@ -85,7 +85,7 @@ TEST_CASE("ApproxCMAESLogisticRegressionTest", "[CMAESTest]")
responses, testResponses, shuffledResponses);
LogisticRegression<> lr(shuffledData, shuffledResponses, 0.5);

ApproxCMAES<> cmaes(0, -1, 1, 32, 200, 1e-3);
ApproxCMAES<> cmaes(0, -20, 20, 32, 200, 1e-3);
arma::mat coordinates = lr.GetInitialPoint();
cmaes.Optimize(lr, coordinates);

Expand Down Expand Up @@ -120,7 +120,7 @@ TEST_CASE("CMAESLogisticRegressionFMatTest", "[CMAESTest]")
responses, testResponses, shuffledResponses);
LogisticRegression<arma::fmat> lr(shuffledData, shuffledResponses, 0.5);

CMAES<> cmaes(0, -1, 1, 32, 200, 1e-3);
CMAES<> cmaes(0, -20, 20, 32, 200, 1e-3);
arma::fmat coordinates = lr.GetInitialPoint();
cmaes.Optimize(lr, coordinates);

Expand Down Expand Up @@ -155,7 +155,7 @@ TEST_CASE("ApproxCMAESLogisticRegressionFMatTest", "[CMAESTest]")
responses, testResponses, shuffledResponses);
LogisticRegression<arma::fmat> lr(shuffledData, shuffledResponses, 0.5);

ApproxCMAES<> cmaes(0, -1, 1, 32, 200, 1e-3);
ApproxCMAES<> cmaes(0, -20, 20, 32, 200, 1e-3);
arma::fmat coordinates = lr.GetInitialPoint();
cmaes.Optimize(lr, coordinates);

Expand Down