Introduction

Welcome to the Bayesian-Optimization-with-Gaussian-Processes wiki!

This is a constrained global optimization package built upon Bayesian inference and Gaussian process, that attempts to find the maximum value of an unknown function in as few iterations as possible. This technique is particularly suited for optimization of high cost functions, situations where the balance between exploration and exploitation is important.

This package was motivated by hyper-parameter optimization of machine leaning algorithms when performing cross validation. Some of the design choices were clearly made with this setting in mind and ultimately a out-of-the-box cross validation optimization object will be implemented (soon).

Disclaimer: This project is under active development, some of its functionalities and syntaxes are bound to change, sometimes dramatically. If you find a bug, or anything that needs correction, please let me know.

---

LAYOUT

BayesianOptimization object

This is the main object in this package. It takes arguments an arbitrary function, and a dictionary of parameters (keys) and bounds for the search for each parameter (values), and attempts to find a global maximum of the target function in an effective way by leveraging exploration and expectation.

BayesianOptimization( f, pbounds, verbose=1 )

Parameters:

• f: A function that returns a real number, e.g.: f = lambda x, y: -x**2 - (y-1)**2
• pbounds: A dictionary with parameters as keys and a tuple with the minimum and maximum value for each during search time, e.g.: pbounds = {'x': (x_min, x_max), 'y': (y_min, y_max)}
• verbose: controls level of verbosity.

Methods:

• initialize(points_dict): A method that allows the user to pass known function values and their corresponding coordinates to the optimizer object prior to the optimization procedure. Usage: self.initialize({'target': [-1, -4], 'x': [0, 2], 'y': [0, 0]})
• explore(points_dict): A method that allows the user to pass a list of points that the optimizer must explore during the procedure. These points will be evaluated at the beginning of the process and can help save time. Usage: self.explore({'x': [-1, 1, 2], 'y': [0, -3, 1]})
• maximize(init_points=5, restarts=50, n_iter=25, acq='ei', \**gp_params): The main method. Takes as parameters the number of random points to initialize the function at. The number of times to restart at random locations when maximizing the acquisition function (greedy). The number of times the algorithm will run. The acquisition function to be used. And any parameters that will be passed to the sklearn Gaussian process object.