Support to configure executor for benchmark

[donglinjy]

Description

Instead of global executor configure, support to set executor at benchmark level.
fix issue: #624

Changes

Adding parameter executor of type orion.executor.base.Executor to run the benchmark experiments.

def get_or_create_benchmark(
    name, algorithms=None, targets=None, storage=None, executor=None, debug=False
):

Checklist

Tests

• I added corresponding tests for bug fixes and new features. If possible, the tests fail without the changes
• All new and existing tests are passing ($ tox -e py38; replace 38 by your Python version if necessary)

Documentation

• I have updated the relevant documentation related to my changes

Quality

• I have read the CONTRIBUTING doc
• My commits messages follow this format
• My code follows the style guidelines ($ tox -e lint)

[bouthilx]

Seems good! Only tests are missing. I think next step after this PR would be to use the executor to parallelize the experiments execution instead of parallelising single experiments sequentially. (All experiments at the same time vs one experiment with parallel workers)

[bouthilx]

Do you think executor should be part of the config? If this affects parallelism and results it may be better to include it.

[donglinjy]

I thought this a little bit. I took it out for 2 consideration:

1. this configure is primary useful when user run the one benchmark on multiple workers(manually create benchmark separately and launch it). It should be okay that at different worker use different executor configure.
2. this configure will be persisted, and now we pass-in a executor object directly instead of the executor configure, to do the persist, need to change this or ask executor return a configure for persistence.

[bouthilx]

Do you think we would need to persist the configuration in the case of the parallelization assessment? I seems to me it would be better to. Maybe that can be another PR, since it would require supporting configuration of the executor through dict arguments instead of simply an instantiated object.

[bouthilx]

Like here: https://github.com/Epistimio/orion/pull/634/files/cb4edf3667330ad6863cfde698f7591b91a281ed..5032cc1fa5c1323d65f6aa0c405530ac9816f617#diff-850efe1c00bd8d6203e80177ccbf6bcc12058cb1d15c8d56b8902728379bf94cR313

[donglinjy]

For parallelization assessment, yes, I think we need to pass-in executor name and configure instead of an instance for persistence, this is because it is part of the benchmark and should not change when we launch same benchmark at different nodes.

[bouthilx]

LGTM! :) Just a few minor comments to address/discuss before ready for merge.