N-way ANOVA wrong results

Hi everyone,

I am doing some statistics functions in Matlab and checking them in SPSS. I am doing N-way ANOVA in Matlab, but it gives me wrong results comparing it to SPSS. As you can see later, the results completely differ, they are not even close. This is my code (I am showing you the brief version of the data to show what is going on):

connection= [0.3950; 0.3774; 0.0622; 0.0140; 0.1424; 0.1029; 0.1711; ...
    0.1595; 0.0774; 0.0111];
abstinence=[12;6;8;11;14;6;3;6;3;4];
relapse=[0;1;2;0;1;1;3;0;1;0];
[~,tbl]=anovan(connection,{abstinence, relapse},'display','on','model','full',...
    'varnames',{'Abstinence','Relapse'}, 'continuous',[1,2]);

I am doing it to be full interaction and with continuous predictors, the same as in SPSS. And this gives me this results in Matlab:

  Source               Sum Sq.   d.f.   Mean Sq.    F     Prob>F
----------------------------------------------------------------
  Abstinence           0.02207    1     0.02207    0.93   0.3714
  Relapse              0.01428    1     0.01428    0.6    0.4667
  Abstinence*Relapse   0.01587    1     0.01587    0.67   0.4441
  Error                0.14196    6     0.02366                 
  Total                0.1653     9                             

And in SPSS (sorry it is in Spanish: Abstinencia is Abstinence and Recaidas is Relapse, just in case):

I don't know why this is happening or if I am doing something wrong. Besides, I have changed 'model' to 'interaction' and it does not work either. Please any help is welcomed!

Thanks in advanced.

Answer 1

[Jeff Miller]

Both of the MATLAB output tables make sense to me, but I don't really know what SPSS is doing.

For the MATLAB continuous case, the model says that connection changes linearly with Abstinence, with Relapse, and with Abstinence*Relapse. The 1 df for each of those terms is a slope relating connection to each of these three numerical predictors. (This is really a regression model.)

For the MATLAB non-continuous case, each different score on Abstinence and Relapse indicates a qualitatively different condition (i.e., imagine that the numbers were just arbitrary labels). Since you have 7 different Abstinence values, this model requires 6 df for Abstinence differences. Similarly you have a lot of distinct Relapse values & a whole lot of possible interaction terms. Basically, you have no where near enough data to estimate the parameters of this model, hence all those zeros in the table.

SPSS is claiming 5 dfs for Abstinence and 2 for Relapse, but I don't know why those aren't 6 & 3 (number of distinct scores minus 1). It has excluded the interaction term completely (0 df), presumably because it recognizes that there are not enough data to estimate those terms.

Bottom line: With your data, it does not seem feasible to use ANOVA as you are trying to do it. This analysis is meant to be used with categorical predictors (i.e., having a few distinct values, which are not assumed to be numerically meaningful), and with lots of data for all possible combinations of values on all the different predictors. For example, you would need several cases with Abstinence=4 & Relapse=0, several with Abstinence=4 & Relapse=1, and so on for all possible cases. I think regression modelling is more suitable for the type of data that you seem to have.

[Elsa Martinez]

Well, I have submit here only a small sample of the real data, I have a long dataset. What happens is that with the complete dataset, Abstinence*Relapse match in both SPSS and Matlab (only in the non-continuous case), but it doesn't in Abstinence and Relapse individually, I don't know why this happens. So I was wondering if you know the difference between SPSS and anovan in Matlab and if I could do something to make them give the same results.

[Elsa Martinez]

I have discovered that the Sum of Squares Type II corresponds with the Sum of Squares Type III in SPSS, so the problem is solved!

But anyways, thanks a lot for your help Jeff!

Answer 2

[Jeff Miller]

The SPSS output does not match up with the data that you posted, so maybe you should check that analysis first. For your connection values, these are the correct SS & df Total values, as produced by MATLAB: 0.1653 9. Since those numbers don't appear in the SPSS output table, I am wondering whether VAR00005 is some other variable. Even if the programs were fitting slightly different models, MATLAB's SS total and SPSS's Total corrigedo should be identical if the data values were the same.

[Elsa Martinez]

I am sorry, you are right, I got confused uploading the image, but the problem is still there with the correct results.

If I have this code I have the following results (I don't put them as continuous):

[~,tbl]=anovan(connection,{abstinence, relapse},'display','on','model','interaction',...
    'varnames',{'Abstinence','Relapse'});
  Source               Sum Sq.   d.f.   Mean Sq.   F   Prob>F
-------------------------------------------------------------
# Abstinence           0          0     0          0    NaN  
# Relapse              0          0     0          0    NaN  
# Abstinence*Relapse   0          0     0          0    NaN  
  Error                0.03768    1     0.03768              
  Total                0.1653     9                          

But if I put them as continuous:

    Source               Sum Sq.   d.f.   Mean Sq.    F     Prob>F
----------------------------------------------------------------
  Abstinence           0.02207    1     0.02207    0.93   0.3714
  Relapse              0.01428    1     0.01428    0.6    0.4667
  Abstinence*Relapse   0.01587    1     0.01587    0.67   0.4441
  Error                0.14196    6     0.02366                 
  Total                0.1653     9                 
  

These are the correct results in SPSS (sorry for that). As you can see, the error does match with the not continuous data and so does the 'total corregido' (total with correction), but it doen't match with the continuous data. But if I don't put continuous I don't get anything.

I don't get why this is happening. Thanks you for your answer!

[John TS]

I am actually having the same problem. In my case flow rate was a treatment with 3 levels that I tuned during experiments to 10, 15 and 20 l/min.

Here, flow rate is a continuous (random factor) instead of a fixed factor, since the magnitude of the values have order. Both Matlab and SPSS are set to Type III Sum of Squares...Matlab gives the degrees of freedom for flow rate as 1; and SPSS gives 2...Can someone explain this difference. When I make flow rate a fixed factor in both software packages I get the same result, i.e. SPSS and Matlab agree with the same significance and both having d.f. = 2 for flow rate.

[Jeff Miller]

Matlab often treats numerical variables as linear predictor covariates. Hence it gives them only 1 df, corresponding to the slope of the dependent variable against the predictor. I suspect that is what is going on with your flow rate variable.

[John TS]

Jeff Miller, Thanks for the response...makes sense. Maybe improving the documentation on 'continuous' in N-way ANOVA would help. For now I have to rely on SPSS to confirm the Matlab stats results. Also having different results when using kstest() in Matlab for normality, compared to SPSS.

[Adam Danz]

@John TS you could go to the anovan page of the online documentation and scroll to the very bottom where you can give that page a star-rating and after that you have the opportunity to type suggestions for improvement to the documentation. The doc team will get that feedback.

@Jeff Miller , thanks for answering stats questions and offering great advice. Not many contributors here take on the stats questions.

[Jeff Miller]

Thanks, Adam, I appreciate the feedback.