Getting NaN after assigning a squared value to a vector

I have been implementing a script lately and I came across problem that I can't solve at all. I have been doing some equation in the loop and assigning the result to a vector element afterwards. To be more precise - I was squaring a double and trying to put it into a vector. I have checked the values I wanted to square with a breakpoint and it was just an ordinary double value. The problem appeared when I was trying to assign it's square to a vector element - when I've done that the element was showing me NaN instead of the actual squared double value.

Here's a part of my code, I think it is (now) sufficient.

I have updated the code after some community feedback to make the problem more clear

% initial values
X  = [1 0 0; 1 1 1; 1 0 1; 1 1 0]; % first column if for bias
Y  = [0 1 0 0];
% These *doubles* are passed to the function: w_min, w_max, learn_coeff, theta
% some are randomly generated:
bias = rand * (w_max - w_min) + w_min;
w1 = rand * (w_max - w_min) + w_min;
w2 = rand * (w_max - w_min) + w_min;
W = [bias w1 w2]; 

% fun part starts here
    err = ones([4 1]); % we start with huge error
    goal = repmat(0.1,[4 1]); % we aim to have error smaller than ...
    iter = 1; 
    tic; 
    while 1
          % net = inputs * weights
          net = X(iter,:) * W';
          % error assignment
          err(iter) = (Y(iter) - net)^2;
          disp(isnan(err(iter)));
          % weights correction
          W = W + learn_coeff * err(iter) * X(iter,:);
          % error checking at the end of an epoch
          if iter == 4 && isequal((err <= goal),true([4 1]))
                weights = W;
                time = toc;
                break;
          end
          iter = 1 + mod(iter,4);
    end

As you can see nothing fancy is really happening there but nevertheless I get a problem, to be precise in this line:

err(iter) = (Y(iter) - net)^2;

I have tried to assign this square also in other ways, playing with brackets, simple multiplication etc etc. Getting NaN all the time even though the squared value (Y(iter) - net) is always double.

tl;dr: Why do I get NaN in the following part?

disp(isnan(err(iter)));

PS: This might be an obvious on but I'm not an experienced Matlab user.

PS2: I have changed the code a bit after getting some responses - so if you've been here already, take another peek :D

[John D'Errico]

wmax = ?
wmin = ?

[Jakub Amanowicz]

These are boundaries of randomly generated weights. W.r.t. the following snippet

% These *doubles* are passed to the function: w_min, w_max, learn_coeff, theta
% some are randomly generated:
bias = rand * (w_max - w_min) + w_min;
w1 = rand * (w_max - w_min) + w_min;
w2 = rand * (w_max - w_min) + w_min;
W = [bias w1 w2];

[Jakub Amanowicz]

I think I've discovered what was the issue - it was purely algorithmic.

If you look at the code you can see that the weight vector depends on error, the error depends on the net value and then the net value depends on weight vector again. This circular dependency is completely valid in terms of the task I'm trying to solve (Adaline neuron) but the way I implemented it is (was) wrong. I have defined error as the squred difference between the expected output and the net value - it shouldn't be squared. Rising it to the power of 2 ended up in producing positive value all the time.

Considering the fact that learning coefficient and the input are both positive we get constantly increasing value of weights which, as mentioned before, influences the net value which influences the error value. At some point it just got overloaded, hence the NaN situation.

See the part of the loop below:

 net = X(iter,:) * W';
 err(iter) = (Y(iter) - net)^2; % here it gets messy because of the ^2
 W = W + learn_coeff * err(iter) * X(iter,:);

Answer 1

[John D'Errico]

There are lots of computations that can generate a NaN.

0/0
inf - inf
inf/inf
0*inf

I imagine I can come up with others if I try.

We don't have any information on what your variables contain, so it is impossible to know what you are doing, or why this is going to NaNs. So, I'm sorry but no, your code was insufficient. If you want a complete answer, then post the values of the variables X, Y, W, learn_coeff, etc. Whatever variables are need to run this code block. Do it as a .mat file. Attach it to a comment.

By the way, it is a REALLY BAD idea to use the variable name diff. diff is a tremendously valuable function. But if you use it as a variable, expect one day to have a bug in your code. Then you will post an anguished question that asks why diff no longer works.

[Jakub Amanowicz]

Hey John, thanks for answering!

I don't believe there is a possibility to divide by 0 or using an inf in my code so it is probably something else causing the problem. The most 'edgy' arithmetic operation there might be 0^2 which is perfectly valid thing to do anyway.

I have changed the question a little to provide other values that are present in the equations. You can also find the .m file in the attachment.

I did not thought about the diff function. Anyway, I used it only for the case of this question so it didn't really influence the behavior. Thank you for the heads up though :)

[John D'Errico]

The common functions I see overloaded by variables are sum, diff, alpha, beta, gamma. But in general, just automatically think if a name is a function before you create any variable.

My guess is you have a variable that went to inf, as an overflow. inf will propagate itself. At some point, you end up with an inf-inf situation, a difference between infs. That yields NaN.

Simplest is just to do as Steve suggested. Set

dbstop if infnan

Then execute your function. I can't do that, because there are still unknown variables. But when the debugger kicks out, look at the line that created the problem. Look at the variables in that expression.

My style of debugging tends not to use dbstop, unless the code is too lengthy, or runs too many iterations. I'd just step through the code one line at a time with the debugger. I'd watch for things starting to head towards a problem. I'll set a break point at various important lines inside loops, then jump ahead to the next breakpoint.

Edit: I see in your response to Steve that you did something of the sort. But since I know that expression does not create a NaN with those variable values, I positively know that something else is happening. I can't test the code yet though.

[Jakub Amanowicz]

Please take a look at the comment I have made to the original question - I think I found the bug.

I rally appreciate your time and effort :) Thank you

Answer 2

[Steven Lord]

You could set an error breakpoint to stop execution as soon as MATLAB encounters an Inf or NaN value. Once you reach that breakpoint, examine the values of variables in the workspace. Of the expressions John posted that could result in NaN, my guess is that you're computing Inf - Inf. If that's true you'll stop when the first of those Inf values gets computed.

[Jakub Amanowicz]

Thank you for suggestion.

I have done the debugging and examine all the variables that might have caused the problem but they were just doubles.

If you take a look at the code above, this line is causing the NaN behavior:

err(iter) = (Y(iter) - net)^2;

The sample values that we deal here with at the moment when the problem occurred:

Y(iter) = 0

net = 0.770306580475225

So I would actually expect to see err(iter) set to 0.5934 rather than NaN