Hyperspaces
HyperSpace works by dividing up the hyperparameter search space into smaller subspaces, running the optimization in parallel over the many subspaces. Here is an example of how the search space is divided with two hyperparameters:
If you think about the potential settings of hyperparameters as an array, we can then divide each hyperparameter into two equally sized subarrays. The parameter phi represents a percent overlap between the two subarrays. Once the hyperparameter arrays are divided, we then create all possible combinations of the subarrays. Each of these combinations of subarrays are what we call a hyperspace, and we run the optimization procedure over the many hyperspaces in parallel. We therefore run 2^H parallel optimization procedures, where H is the number of hyperparameters.