[Bug] optimize_acqf_cyclic fails due to batch_initial_conditions shape when used on a problem with inequality constraints. #873

alan-tan-wm · 2021-07-21T05:05:14Z

🐛 Bug

I'm running optimize_acqf_cyclic on a problem with inequality constraints and it seems to be failing in _make_linear_constraints. It seems that the batch_initial_conditions=candidates[i].unsqueeze(0) (line 301) is of shape 1 x dims, but _make_linear_constraints is expecting something of shape b x q x d.

candidates[i].unsqueeze(0).unsqueeze(0) seems to work.

** Stack trace/error message **

File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\optimize.py", line 309, in optimize_acqf_cyclic
candidate_i, acq_val_i = optimize_acqf(
File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\optimize.py", line 184, in optimize_acqf
batch_candidates_curr, batch_acq_values_curr = gen_candidates_scipy(
File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\generation\gen.py", line 132, in gen_candidates_scipy
constraints = make_scipy_linear_constraints(
File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\parameter_constraints.py", line 119, in make_scipy_linear_constraints
constraints += _make_linear_constraints(
File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\parameter_constraints.py", line 223, in _make_linear_constraints
raise UnsupportedError("shapeX must be b x q x d")
botorch.exceptions.errors.UnsupportedError: shapeX must be b x q x d

Expected Behavior

I'm expecting to be able to use optimize_acqf_cyclic on a problem with inequality constraints.

System information

Please complete the following information:

0.5.1.dev14+g951483dc
1.5.0
1.9.0+cu111
Windows 10

Additional context

Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

Balandat · 2021-07-21T14:40:35Z

@sdaulton could you take a look at this one, please?

Balandat · 2021-09-19T16:48:46Z

@alantan-SQ40 could you please provide a full repro so we can debug? Thanks!

alan-tan-wm · 2021-09-20T04:23:48Z

OK. Here's an example:

from botorch.utils.testing import MockAcquisitionFunction
from botorch.optim.optimize import optimize_acqf_cyclic
from torch import tensor
bounds = tensor([[0., 0., 0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1., 1., 1.]], device='cuda:0',
       dtype=torch.float64)
ineq_constraints = [
            (tensor([0, 1], device='cuda:0'), tensor([ 1., -1.], device='cuda:0', dtype=torch.float64), tensor(-0., device='cuda:0', dtype=torch.float64))
        ]
optimize_acqf_cyclic(acq_function=MockAcquisitionFunction(),
                bounds=bounds,
                q=3,
                inequality_constraints=ineq_constraints,
                fixed_features=None,
                post_processing_func=None,
                num_restarts=2,
                raw_samples=10,
                cyclic_options={'maxiter':2}     
                     )

It results in:

[INFO 09-20 12:20:32] botorch: Generated candidate batch 1 of 1.
[INFO 09-20 12:20:32] botorch: Generated sequential candidate 1 of 3
[INFO 09-20 12:20:33] botorch: Generated candidate batch 1 of 1.
[INFO 09-20 12:20:33] botorch: Generated sequential candidate 2 of 3
[INFO 09-20 12:20:34] botorch: Generated candidate batch 1 of 1.
[INFO 09-20 12:20:34] botorch: Generated sequential candidate 3 of 3
Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2021.1.1\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_exec2.py", line 3, in Exec
    exec(exp, global_vars, local_vars)
  File "<string>", line 10, in <module>
  File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\optimize.py", line 309, in optimize_acqf_cyclic
    candidate_i, acq_val_i = optimize_acqf(
  File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\optimize.py", line 184, in optimize_acqf
    batch_candidates_curr, batch_acq_values_curr = gen_candidates_scipy(
  File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\generation\gen.py", line 132, in gen_candidates_scipy
    constraints = make_scipy_linear_constraints(
  File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\parameter_constraints.py", line 119, in make_scipy_linear_constraints
    constraints += _make_linear_constraints(
  File "C:\Anaconda3\envs\ai-engine-updated\lib\site-packages\botorch\optim\parameter_constraints.py", line 223, in _make_linear_constraints
    raise UnsupportedError("`shapeX` must be `b x q x d`")
botorch.exceptions.errors.UnsupportedError: `shapeX` must be `b x q x d`

Tested on BoTorch version 0.5.1.
Does that help?

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: f19c01ca7ed8fa759b63000189ffa9974248dadb

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: dd17e7aa46b309e02c8cfde359c8a98cb102973f

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: 8461b2b4a724a4961cf4781501a55149281dad9b

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: dea041dde53f9db51020167e3d940ab0473a6436

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: a9fd5afa049e912a94d0b3bae36e33e5deddc32a

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before [x] Make `_make_linear_constraints` work with 2d inputs so that `optimize_acqf` also does (previously, optimize_acqf only worked in some cases) Reviewed By: Balandat Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: b68eb33ed36b5d1ec3c2d5264283ecb128fe6545

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before [x] Make `_make_linear_constraints` work with 2d inputs so that `optimize_acqf` also does (previously, optimize_acqf only worked in some cases) Reviewed By: Balandat Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: 7d2519cf7ecd509c377e09e2c1adfb3efbb84e3e

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before [x] Make `_make_linear_constraints` work with 2d inputs so that `optimize_acqf` also does (previously, optimize_acqf only worked in some cases) Reviewed By: Balandat Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: a4faed4245d2638fd02189d2657ab42a03127bb1

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before [x] Make `_make_linear_constraints` work with 2d inputs so that `optimize_acqf` also does (previously, optimize_acqf only worked in some cases) Reviewed By: Balandat Differential Revision: D42875942 Pulled By: esantorella fbshipit-source-id: 15f5ad47e682d24c7a7ff5efff5b9a9727839b45

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before [x] Make `_make_linear_constraints` work with 2d inputs so that `optimize_acqf` also does (previously, optimize_acqf only worked in some cases) Differential Revision: https://internalfb.com/D42875942 Pulled By: esantorella fbshipit-source-id: 274a27da954847fbd278a76d20bf42d5be6b4872

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before [x] Make `_make_linear_constraints` work with 2d inputs so that `optimize_acqf` also does (previously, optimize_acqf only worked in some cases) Differential Revision: https://internalfb.com/D42875942 Pulled By: esantorella fbshipit-source-id: 5eab158fda510241b9c4df82d7f210d4777a6e1a

Summary: ## Motivation Fixes pytorch#873 In the past, `optimize_acqf` implicitly needed 3d inputs when there are equality constraints or inequality constraints and fixed_features don't provide the trivial solution, even though it worked with 2d inputs (no b-batches) in other cases. `optimize_acqf_cyclic` passed it 2d inputs, which would not generally work. I initially considered changing `optimize_acqf_cyclic` to pass 3d inputs, but since I found another place where 2d inputs were used, I decided to change `optimize_acqf` so it works with 2d inputs instead. This was not caught because the only usage of `optimize_acqf_cyclic` was in a test that mocked `optimize_acqf`, so `optimize_acqf_cyclic` was never actually run end-to-end. I changed the test for `optimize_acqf_cyclic` to be more end-to-end, at the cost of worse testing of some intermediate properties. We could keep both versions though. [x] Better docstring documentation on input shapes [x] Add a singleton leading b-dimension where initial conditions are 2d Pull Request resolved: pytorch#1648 Test Plan: [x] More end-to-end test of `optimize_acqf_cyclic` that doesn't stub in `optimize_acqf` (see above) [x] more input validation and unit tests for input validation [x] Ran cases that now raise errors without the new error handling, to make sure they were erroring before [x] Make `_make_linear_constraints` work with 2d inputs so that `optimize_acqf` also does (previously, optimize_acqf only worked in some cases) Differential Revision: https://internalfb.com/D42875942 Pulled By: esantorella fbshipit-source-id: 6d5b69a23810578e0fb5bd5c9814b774c3e1c45d

alan-tan-wm added the bug Something isn't working label Jul 21, 2021

Balandat assigned sdaulton Jul 21, 2021

esantorella self-assigned this Jan 30, 2023

esantorella mentioned this issue Jan 30, 2023

Fix shape error in optimize_acqf_cyclic #1648

Closed

facebook-github-bot closed this as completed in 2f2b7e2 Feb 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] optimize_acqf_cyclic fails due to batch_initial_conditions shape when used on a problem with inequality constraints. #873

[Bug] optimize_acqf_cyclic fails due to batch_initial_conditions shape when used on a problem with inequality constraints. #873

alan-tan-wm commented Jul 21, 2021

Balandat commented Jul 21, 2021

Balandat commented Sep 19, 2021

alan-tan-wm commented Sep 20, 2021

[Bug] optimize_acqf_cyclic fails due to batch_initial_conditions shape when used on a problem with inequality constraints. #873

[Bug] optimize_acqf_cyclic fails due to batch_initial_conditions shape when used on a problem with inequality constraints. #873

Comments

alan-tan-wm commented Jul 21, 2021

🐛 Bug

Expected Behavior

System information

Additional context

Balandat commented Jul 21, 2021

Balandat commented Sep 19, 2021

alan-tan-wm commented Sep 20, 2021