Improvements of Random Search for Hyperparameter Optimization

Hello

Random search is one possibility for hyperparameter optimization in machine learning. I have applied random search to search for the best hyperparameters of a SVM classifier with a RBF kernel. Additional to the continuous Cost and gamma parameter, I have one discrete parameter and also an equality constraint over some parameters.

Now, I would like to develop random search further, e.g. through adaptive random search. That means for example adaptation of the search direction or of the search range.

Does somebody have an idea how this can be done or could reference to some existing work on this? Other ideas for improving random search are also welcome.

Answer 1

[Don Mathis]

You might have a look at the bayesopt function in the R2016b release.