How to make a function to initialize weights in MATLAB using Nguyen-Widrow algorithm for a 47-H-1 multilayer perceptron?

I'm looking for a way to make a function to initialize weights in MATLAB using Nguyen-Widrow algorithm for a 47-H-1 multilayer perceptron. This must be accomplished without libraries using just matricies.

Answer 1

[Shubham]

Hi Sweet,

When initializing weights in a neural network using the Nguyen-Widrow algorithm, the goal is to set the initial weights and biases such that the neurons in the hidden layer start with activation potentials that are spread over the widest possible area of the activation function. This is particularly useful for networks using sigmoidal activation functions.

The Nguyen-Widrow initialization algorithm follows these steps:

1. First, the weights between the input layer and the hidden layer are initialized to small random values, typically in the range of [-1, 1]. Similarly, the biases for the hidden layer neurons are also initialized randomly within the same range.
2. The Nguyen-Widrow algorithm uses a constant, beta, which is calculated using the formula where H is the number of hidden neurons and I is the number of input neurons.
3. Each weight vector for the hidden neurons is normalized. This means that for each hidden neuron, you calculate the norm (length) of its weight vector (including the bias as an additional dimension), and then you adjust the weights such that the norm of the new weight vector is equal to the beta value calculated in the previous step.
4. The biases for the hidden layer are scaled in the same way as the weights, so that the length of the weight vector (including the bias) matches the beta value.
5. The weights and biases for the output layer are typically initialized to small random values, as the Nguyen-Widrow algorithm is primarily concerned with the initialization of the first layer of weights.

Here is the documentation link for Nguyen-Widrow algorithm:https://in.mathworks.com/help/deeplearning/ref/initnw.html

I hope it helps!