Bayesianoptimization algorithm does not work when hyperparameter are negative doubles

[JustinReboullot]

/kind bug

What steps did you take and what happened:

Katib suggest NaN values when submitting an experiment with:

• a parameter with parametertype: double
• a feasibleSpace including negative numbers
• bayesianoptimization as algorithm

For example this kind of yaml makes Katib suggesting NaN for the parameter my-param.

apiVersion: "kubeflow.org/v1alpha3"
kind: Experiment
metadata:
  namespace: kubeflow
  labels:
    controller-tools.k8s.io: "1.0"
  name: my-buggy-experiment
spec:
  objective:
    type: maximize
    goal: 0.99
    objectiveMetricName: my_metric
  algorithm:
    algorithmName: bayesianoptimization
    algorithmSettings:
      - name: "n_initial_points"
        value: "5"
  metricsCollectorSpec:
    collector:
      kind: StdOut
  parallelTrialCount: 1
  maxTrialCount: 2
  maxFailedTrialCount: 3
  parameters:
    - name: --my-param
      parameterType: double
      feasibleSpace:
        min: "-1.0"
        max: "1.0"
trialTemplate:
    goTemplate:
        rawTemplate: |-
          apiVersion: batch/v1
          kind: Job
          metadata:
            name: {{.Trial}}
            namespace: {{.NameSpace}}
          spec:
            template:
              spec:
                containers:
                - name: {{.Trial}}
                  image: some_image
                  command:
                  - "python"
                  - "some_script.py"
                  {{- with .HyperParameters}}
                  {{- range .}}
                  - "{{.Name}}={{.Value}}"
                  {{- end}}
                  {{- end}}
                restartPolicy: Never

What did you expect to happen:
Katib should not suggest NaN for my-param but values between -1 and 1.

Anything else you would like to add:
Is it due to the log-uniform prior in https://github.com/kubeflow/katib/blob/489d52b0094e36ccb53d26cf0127add9e00bd606/pkg/suggestion/v1alpha3/skopt/base_skopt_service.py line 43?

Environment:

• Kubeflow version: 0.7.0
• Minikube version:
• Kubernetes version: (use kubectl version): 1.14.0
• OS (e.g. from /etc/os-release): Ubuntu 18.04.3 LTS

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
bug	0.99

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[andreyvelich]

According to this https://scikit-optimize.github.io/space.m.html#skopt.space.Real, we can specify uniform there.
What do you think @gaocegege ?

[janvdvegt]

This is biting us as well, it's even when the double space starts at 0.

[JustinReboullot]

According to the code, the only supported way to use bayesian optimisation with real number parameters is by searching for them in log scale. That is to say if your feasibleSpace goes from x_min to x_max, katib will search for an optimal value of x between log_10(x_min) and log_10(x_max). If that is acceptable for you can just consider this as a feature and set 0 < x_min <= x_max.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreyvelich]

/lifecycle frozen

[andreyvelich]

That should be fixed as part of: #1207.