How can two neural networks be compared for regression based on training and testing results ?

How can two neural networks be compared for regression based on training and testing results ?

[Greg Heath]

Since it is obvious that 2 nets can be compared by plotting their reponses, it is unclear what your problem is.

Please elucidate.

Greg

[Kangujam]

@Greg Heath actually after implementation of neural network for regression, training and testing MSE results will be obtained. So from those neural networks, which mse has to be chosen for comparison? Will it be training mse or testing mse?

Answer 1

[BERGHOUT Tarek]

for regression the lower error the greater accuracy is the network gets . you can also use a T test for you output analysis to determine which net is better

Answer 2

[Greg Heath]

The MATLAB default is training/validation/testing fractions of 0.7/0.15/0.15

Typically, the performance depends on a

1. A reasonable choice for number of hidden layers and nodes

2. A successful choice of RANDOM division into train/val/test subsets

3. A successful group of RANDOM initial weights

MY APPROACH:

1. A single hidden layer

2. Loop over 0 to Hmax trial values for numHidden

3. 10 random initial weight trials for each test value of H

4. MSEgoal = 0.01*mean(var(target',1))

NETWORK GRADING

grade = alpha*MSEtst + beta*MSEval

If N is sufficiently large alpha = 1, beta = 0