fcmaes gives you a clear advantage at an optimization competition like ESAs optimization challenges if you exploit its parallelization capabilities. Here is a list of the ranks achieved by fcmaes fcmaes:
For most challenges the fcmaes result is either the best or less than 1 % behind. All competition code was written in Python - although fcmaes and or-tools wrap C++ code to improve performance. All these 15 challenges are still open, you can submit solutions and you result will show at the leaderboard.
In this domain we often have multiple competing objectives and a lot of constraints. We present results for the Mazda real-world car structure design benchmark, the simultaneous optimization of three car models minimizing their weight, increasing the number of shared thicknesses of structural parts thereby fulfilling 54 constraints. 2017 there was a competition related to this problem Report of Evolutionary Computation Competition 2017, but until now not many of the ideas produced there have found their way into open source optimization libraries.
We applied modecpp.py for about 1 hour runtime using one AMD 5950x CPU on Linux, de/rand/1 strategy (nsga_update=False, pareto_update=False, ints=[True]*dim), population size = 256. We choose the best run out of two executed in parallel, each utilizing 16 threads (workers=16). This way about 8200 function evaluations are performed per second for both runs combined.
The resulting pareto front with hypervolume 0.4074 is:
The "reference" NSGA-II solution given as part of the benchmark has hypervolume 0.1456:
Note, that the reference solution was computed using a limited budget. But NSGA-II scales much worse than fcmaes-MoDe using enhanced multiple constraint ranking.
Remark: This section was initially created in 2022 and updated in 2024. What changed:
-
New hardware: CPU AMD 9950x replaced an AMD 5950x
-
Hardware performance: The 9950x is about factor 1.75 faster than the 5950x for these benchmarks.
-
Random generator: fcmaes switched from Mersenne Twister to PCG64 DXSM both for Python and C++.
fcmaes provides fast parallel example solvers for the real world space flight design problems GTOP and for the F-8_aircraft problem based on differential equations. On GTOPX you can find implementations of the corresponding objective functions using different programming languages. The solution times given in the tables below are for Linux / AMD 9950x CPU utilizing 32 parallel processes.
| problem | runs | absolute best | stopVal | success rate | mean time | sdev time |
|---|---|---|---|---|---|---|
Cassini1 |
100 |
4.9307 |
4.95535 |
100% |
0.69s |
1.1s |
Cassini2 |
100 |
8.383 |
8.42491 |
100% |
12.05s |
5.73s |
Gtoc1 |
100 |
-1581950 |
-1574080 |
100% |
11.11s |
8.49s |
Messenger |
100 |
8.6299 |
8.673 |
100% |
13.33s |
8.16s |
Rosetta |
100 |
1.3433 |
1.35 |
100% |
17.49s |
6.21s |
Tandem |
100 |
-1500.46 |
-1493 |
81% |
166.92s |
147.87s |
Sagas |
100 |
18.188 |
18.279 |
100% |
1.7s |
0.99s |
Messenger Full |
100 |
1.9579 |
1.96769 |
43% |
1757.94s |
1240.32s |
Messenger Full |
100 |
1.9579 |
2.0 |
79% |
693.02s |
515.71s |
Note that 'stopVal' is the threshold value determining success and 'mean time' includes the time for failed runs. Execute benchmark_gtop.py to reproduce these results. The same optimization algorithm was applied for all problems, using the same parameters both for the optimization algorithm and the coordinated retry / boundary management.
| problem | runs | absolute best | stopVal | success rate | mean time | sdev time |
|---|---|---|---|---|---|---|
Cassini1 |
100 |
4.9307 |
4.931 |
100% |
2.24s |
1.87s |
Cassini2 |
100 |
8.383 |
8.384 |
98% |
37.12s |
20.15s |
Gtoc1 |
100 |
-1581950 |
-1581949 |
100% |
23.13s |
16.23s |
Messenger |
100 |
8.6299 |
8.631 |
100% |
27.28s |
12.48s |
Rosetta |
100 |
1.3433 |
1.344 |
91% |
33.0s |
18.16s |
Tandem |
100 |
-1500.46 |
-1500 |
81% |
175.11s |
148.41s |
Sagas |
100 |
18.188 |
18.189 |
100% |
2.05s |
1.06s |
Because of its famous complexity ESAs 26-dimensional Messenger full problem is often referenced in the literature, see for instance MXHCP paper.
fcmaes solves this problem in less than half an hour using an AMD 9950x.
The Problem models a multi-gravity assist interplanetary space mission from Earth to Mercury. In 2009 the first good solution (6.9 km/s) was submitted. It took more than five years to reach 1.959 km/s and three more years until 2017 to find the optimum 1.958 km/s. The picture below shows the progress of the whole science community since 2009:
The following picture shows 100 coordinated retry runs:
79 out of these 100 runs produced a result better than 2 km/s:
About 2.1*10^6 function evaluations per second were performed which shows excellent scaling of the algorithm utilizing all 16 cores / 32 threads. fcmaesray shows how a 5 node cluster using 96 CPU-cores executing fcmaes coordinated retry performs in comparison.