FMINLBFGS: Fast Limited Memory Optimizer

Quasi newton limited memory BFGS and Steepest decent optimizer for large amount of unknowns

FMINLBFGS is a Memory efficient optimizer for problems such as image registration with large amounts of unknowns, and cpu-expensive gradients.

Supported:
- Quasi Newton Broyden–Fletcher–Goldfarb–Shanno (BFGS).
- Limited memory BFGS (L-BFGS).
- Steepest Gradient Descent optimization.

Advantages:
- Quasi-Newton thus no need to provide a hessian, instead the hessian is updated by analyzing successive gradient vectors instead.
- L-BFGS never explicitly forms or stores the Hessian matrix, which can be quite expensive when the number of dimensions becomes large.
- There is an input option to replace gradient calls during linesearch with normal function calls, if the gradient is cpu-expensive.

This function uses the same interface and options as Matlab functions such as fminunc and fminsearch. But optimset can be replaced by a simple struct when optimization toolbox not available.

Example:
options = optimset('GradObj', 'on' ,'Display', 'iter', 'HessUpdate', 'bfgs', 'GoalsExactAchieve',1);
[x,fval] = fminlbfgs(@myfun, [1 1 1 1 1], options);