Bayesopt constrain objective to positive values

I am using bayesopt to minimize a deterministic function which can give both positive and negative results. However, I want to limit the values to a range of only 0 and above. My plan is to use a coupled constraint to do this. Ideally I would want to use inequality constriant, however from my understanding this is not possible with bayesopt coupled constraints.

My question is whether the magnitude of the coupled contraint impacts the results? To illustrate this I have included two solutions that I am considering using:

Solution A:

if  objective >= 0
    constraint = -1;
else 
    constraint = 1;
end

Solution B:

constraint = -objective;

Would solution B be better (assuming that the magnitude of the constraint value impacts the results)? And what would be the implications if the constraint were to reach a value of 0?

Answer 1

[Hassaan]

To answer your question directly:

• Magnitude of the Coupled Constraint: In general, the magnitude of the constraint function's value does not impact the bayesopt results as long as it is negative (indicating a feasible region) or zero. What matters is whether the value is above zero (infeasible) or not (feasible). The optimizer seeks to find the minimum of the objective function while keeping the constraint function's value less than or equal to zero.
• Solution A: This approach effectively creates a binary switch. If the objective is non-negative, the constraint is set to a feasible value (-1), and if the objective is negative, the constraint is set to an infeasible value (1). This is a clear distinction and easy for the optimizer to interpret.
• Solution B: This approach scales the constraint linearly with the objective. This will indeed give negative values when the objective is positive (feasible regions) and positive values when the objective is negative (infeasible regions). However, the magnitude of the constraint function's value does not provide additional leverage to the optimizer in this context, as bayesopt is not gradient-based and does not use the magnitude of constraint violation to guide the search.
• Implications of Zero Value: If the constraint function reaches zero exactly, it indicates the boundary of the feasible region. The optimizer will treat this as a feasible solution (because it is not greater than zero) but will recognize that it cannot go further in that direction without violating the constraint.

Between Solution A and Solution B, Solution A is a more clear-cut approach for the optimizer to understand, as it acts as a hard switch between feasible and infeasible regions without providing ambiguous gradients. Solution B, on the other hand, might make more sense in a gradient-based optimization algorithm where the magnitude of the violation can guide the search process, but in bayesopt, it does not hold significant advantage.

In both cases, when the constraint value is zero, the optimizer recognizes that it is at the boundary of the feasible region. It's important to note that bayesopt will sample points during the optimization process, and some points may violate the constraint. The algorithm uses these samples to better understand the feasible region and improve its model of the objective function landscape within the feasible space.

Therefore, either approach should be suitable for bayesopt as long as the constraint function properly demarcates the feasible region (non-negative objective values). If you wish to use Solution B, which scales the constraint, that's also acceptable, but it's the sign of the constraint function value that bayesopt uses to determine feasibility, not the magnitude.

------------------------------------------------------------------------------------------------------------------------------------------------

If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.

Professional Interests

• Technical Services and Consulting
• Embedded Systems | Firmware Developement | Simulations
• Electrical and Electronics Engineering

Feel free to contact me.

[Isaiah]

Thank you for clarifying. Just to add on, lets say I want to contrain another value, "val", within a certain range (10 to 20). Based on what you have shared I would assume the the following solutions would also be accepctable, with the "Solution A" being the more clear-cut approach:

% solution A
if (val >= 10 & val <= 20)
    constraint = -1;
else
    constraint = 1;
end

% solution B
if val > 15
    constraint = val - 20;
elseif temperature < 15
    constraint = 10 - val;
else
    constraint = -15;
end

[Hassaan]

You can adjust as per your requirements.

------------------------------------------------------------------------------------------------------------------------------------------------ If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback. Professional Interests Technical Services and Consulting Embedded Systems | Firmware Developement | Simulations Electrical and Electronics Engineering Feel free to contact me.