Main Content

Bayesian Optimization Objective Functions

Objective Function Syntax

bayesopt attempts to minimize an objective function. If, instead, you want to maximize a function, set the objective function to the negative of the function you want to maximize. See Maximizing Functions. To include extra parameters in an objective function, see Parameterizing Functions.

bayesopt passes a table of variables to the objective function. The variables have the names and types that you declare; see Variables for a Bayesian Optimization.

The objective function has the following signature:

[objective,coupledconstraints,userdata] = fun(x)

  1. objective — The objective function value at x, a numeric scalar. bayesopt returns an error if the objective function returns a nonnumeric value or a matrix with more than one entry.

  2. coupledconstraints — Value of coupled constraints, if any (optional output), a vector of real values. A negative value indicates that a constraint is satisfied, a positive value indicates that it is not satisfied. For details, see Coupled Constraints.

  3. userdata — Optional data that your function can return for further uses, such as plotting or logging (optional output). For an example, see Bayesian Optimization Plot Functions.

Objective Function Example

This objective function returns the loss in a cross-validated fit of an SVM model with parameters box and sigma. The objective also returns a coupled constraint function that is positive (infeasible) when the number of support vectors exceeds 100 (100 is feasible, 101 is not).

function [objective,constraint] = mysvmfun(x,cdata,grp)
SVMModel = fitcsvm(cdata,grp,'KernelFunction','rbf',...
    'BoxConstraint',x.box,...
    'KernelScale',x.sigma);
objective = kfoldLoss(crossval(SVMModel));
constraint = sum(SVMModel.SupportVectors) - 100.5;

To use the objective function, assuming that cdata and grp exist in the workspace, create an anonymous function that incorporates the data, as described in Parameterizing Functions.

fun = @(x)mysvmfun(x,cdata,grp);
results = bayesopt(fun,vars) % Assumes vars exists

Objective Function Errors

bayesopt deems your objective function to return an error when the objective function returns anything other than a finite real scalar. For example, if your objective function returns a complex value, NaN, or Inf, then bayesopt deems that your objective function errors. If bayesopt encounters an error, it continues to optimize, and automatically updates a Bayesian model of points that lead to errors. This Bayesian model is the Error model. bayesopt incorporates the Error model as a coupled constraint. See Coupled Constraints.

When errors exist, you can plot the Error model by setting the bayesopt PlotFcn name-value argument @plotConstraintModels. Or you can retrospectively call plot on the results of a Bayesian optimization, and include @plotConstraintModels.

Related Topics