How to optimize the hyperparameter for support vector regression with LIBSVM library on MATLAB?

To make a regression model using LIBSVM library on MATLAB? Therefore paraameter of optimization is required. I optimized parameter but it's not working because epsilon does not change with other parameter?

my loop for optimization is such as

bestcv = inf;

for log2c = -1:10

for log2g = -1:5

for log2p = -10:-7

cmd = [ '-v 5 -s 3 ', '-c ', num2str(2^log2c), ' -g ', num2str(2^log2g), '-p', num2str(2^log2p)];

cv = svmtrain(Y, X, cmd);

if (cv<= bestcv)

bestcv = cv; bestc = 2^log2c; bestg = 2^log2g; bestp = 2^log2p;

end

fprintf('%g %g %g %g(best c=%g, g=%g, p=%p rate=%g)\n', log2c, log2g, log2p, cv, bestc, bestg, bestp, bestcv);

end

end

end

How to change in the loop and get correct values? please explain it

Thanks in advance

Answer 1

[surya venu]

Hi,

The provided MATLAB code appears to be performing parameter tuning for a Support Vector Regression (SVR) model using the LIBSVM library. The loop iterates through various combinations of C (cost) and gamma parameters to identify the optimal configuration based on cross-validation results. However, it's important to note that epsilon (epsilon) in SVR is typically a fixed hyperparameter, and it might not be necessary to include it in the optimization loop.

Here is the updated code:

bestcv = inf; % Best cross-validation error initialized to infinity
bestc = 0; % Initialize best C
bestg = 0; % Initialize best gamma
bestp = 0; % Initialize best epsilon

% Loop over the range of parameters
for log2c = -1:10
    for log2g = -1:5
        for log2p = -10:-7
            % Construct the command string with spaces correctly placed
            cmd = sprintf('-v 5 -s 3 -c %f -g %f -p %f', 2^log2c, 2^log2g, 2^log2p);
            
            % Perform cross-validation
            cv = svmtrain(Y, X, cmd);
            
            % Update the best parameters if the current CV error is lower
            if (cv < bestcv)
                bestcv = cv; 
                bestc = 2^log2c; 
                bestg = 2^log2g; 
                bestp = 2^log2p;
            end
            
            % Print the current parameters and the best ones found so far
            fprintf('%g %g %g %g (best c=%g, g=%g, p=%g, rate=%g)\n', log2c, log2g, log2p, cv, bestc, bestg, bestp, bestcv);
        end
    end
end

Here are the changes:

• Corrected the cmd string formatting to ensure proper spacing around parameters.
• Initialized bestc, bestg, and bestp before the loop.

Hope it helps.