How to set the beta coefficient of generation strategy?

[omrisch]

I want to generate explore/exploit experimental suggestions based on different beta parameters.
How can I specify this in the generation strategy?

[bernardbeckerman]

@omrisch thanks for asking this! I'm following up internally to find the right person to help you here.

[Balandat]

Can you provide some more context? I assume by beta you are referring to the exploration component in UCB algorithms / acquisition functions? Are you already using a UCB-type algorithm? Note that by default Ax uses an Expected Improvement-based acquisition function, not UCB.

[omrisch]

Thanks Max, I am using GPEI, and I am wondering if it is possible to control the explore/exploit tendency of the acquisition function, as seen in the following photo: 

…

On Mon, Jun 17, 2024 at 6:58 PM Max Balandat ***@***.***> wrote: Can you provide some more context? I assume by beta you are referring to the exploration component in UCB algorithms / acquisition functions? Are you already using a UCB-type algorithm? Note that by default Ax uses an Expected Improvement-based acquisition function, not UCB. — Reply to this email directly, view it on GitHub <#2525 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOGGGRJIZWKZ7DSOEFCXJYDZH4BUHAVCNFSM6AAAAABJOA2OK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZTG44DINJYHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Balandat]

@omrisch looks like your photo didn't come through (probably b/c you responded via email and not on github).

There is no explicit way to control the explore-exploit tradeoff; for (non-noisy) EI a manual way to achieve this is to artificially deflate the value f* of the incumbent (best observed point so far). Are your observations noiseless? If not, are you providing observation noise variances or are you letting Ax infer the noise level?

This discussion is relevant in this context: pytorch/botorch#373

[omrisch]

My observations are noisy, currently I let Ax infer the noise

[Balandat]

So by default this will use botorch's qLogNoisyExpectedImprovement acquisition function. This implementation does not use an explicit incumbent (since observations are noisy), but instead integrates out the uncertainty in incumbent by sampling its values from the posterior (see sec A.8 in our logEI paper). What that means is that there is no parameter that directly adjusts the exploration tradeoff. One could introduce one, but that would require some thought and research.

You could instead specify a non-noisy acquisition function that uses an explicit incumbent and register a new acquisition function and implement an input constructor for it following the general setup as in our modular BoTorch model tutorial, but my strong expectation is that this will likely result in worse performance compared to qLogNEI.

Do you have any specific concerns that make you think you need to adjust the exploration component?

[mgarrard]

@omrisch closing due to inactivity, please feel free to re-open or start a new issue for additional help :)