I am using Regression Learner App for training the Neural Network, I want to know how to choose the different activation functions for different layers?

Is there any way we can specify different activation functions for each layer in regression learner app.

Answer 1

[Abhipsa]

Hi @SWAROOP H P,

I understand that you want to choose different activations for different layers of neural network in Regression Learner App.

The Regression Learner App allows you to choose the activation function which can be applied to the fully connected layers, except for the final full connected layer. As per the requirements, you can choose one of “ReLu”,” Tanh” and “Sigmoid” activation function.

In addition to this, other workaround would be to use the "Deep Learning Toolbox" in MATLAB.

You can refer to the below code to define a neural network with distinct activation functions for different layers:

layers = [
    featureInputLayer(10) % Input layer for 10 features
    fullyConnectedLayer(50) % First hidden layer with 50 neurons
    reluLayer % ReLU activation for the first hidden layer
    fullyConnectedLayer(20) % Second hidden layer with 20 neurons
    tanhLayer % Tanh activation for the second hidden layer
    fullyConnectedLayer(1) % Output layer with 1 neuron (for regression)
    regressionLayer]; % Regression output layer

options = trainingOptions('adam', ...
    'MaxEpochs', 50, ...
    'MiniBatchSize', 32, ...
    'InitialLearnRate', 0.001, ...
    'Plots', 'training-progress');

net = trainNetwork(trainingData, layers, options);

You can refer the below MATLAB Documentation for more details:

https://www.mathworks.com/help/releases/R2023b/stats/choose-regression-model-options.html

I hope this will help.

[SWAROOP H P]

Thanks for the answer, this helps!!