sensitivity analysis ANN

how to carry out sensitivity analysis i.e. effect of an input parameter on output parameter, in artificial neural network tool box

[Carlos M. Velez S.]

I recommend this Matlab code for implementation of Monte Carlo method for sensitivity analysis of Simulink models: http://www.mathworks.com/matlabcentral/fileexchange/47758-sensitivity-analysis-in-simulink-models-with-monte-carlo-method

Answer 1

[Greg Heath]

Please clarify your terminology. What are some examples of input parameters? Ditto for output parameters.

For sensitivity analysis of input variables and weights:

Standardize (zero-mean/unit-variance) training inputs and use the original means and standard deviations to normalize the validation and test sets.

Repeat the process for regression net outputs.

For pattern recognition nets with c classes, use one-of-c binary normalization (each column of the target matrix is a column of the c-dimensional unit matrix).

The easiest way to test variable sensitivity is to randomly reorder the row for the variable in question and obtain the resulting summary statistics (e.g, min/median/mean/std/max or quartiles) of the chosen MOP (i.e., measure of performance ; e.g., normalized mean-squared error, NMSE or the coefficient of determination R^2). Training data MOPs should be degree-of-freedom adjusted to mitigate the optimistic bias. Validation and test set estimates are assumed to be unbiased.

The easiest way to test weight sensitivity is to repeatedly randomize the weight in question and obtain the resulting summary statistics.

These techniques have been used iteratively to eliminate ineffective variables and/or weights. As usual for greedy searches, the results are suboptimal but are obtained relatively quickly.

Hope this helps.

Greg

[Arso]

Hi all,

the solution is here http://www.sciencedirect.com/science/article/pii/S0169260799000462 - you will have to code it by yourself. Statistica has implemented this features, Matlab implementation of ANN is missing a lot of options for performing serious analysis.

I think that during the last years evolution of Matlab (which is commercal) has relid on the community too much - in the most cases codes provided by users were more usefull for me than those provided by toolboxes.

best,

Arso V.