multcompare

Multiple comparison of estimated marginal means

Syntax

tbl = multcompare(rm,var)

tbl = multcompare(rm,var,Name,Value)

Description

tbl = multcompare(rm,var) returns multiple comparisons of the estimated marginal means based on the variable var in the repeated measures model rm.

tbl = multcompare(rm,var,Name,Value) returns multiple comparisons of the estimated marginal means with additional options specified by one or more Name,Value pair arguments.

For example, you can specify the comparison type or which variable to group by.

Examples

Multiple Comparison of Estimated Marginal Means

Load the sample data.

load fisheriris

The column vector species consists of iris flowers of three different species: setosa, versicolor, and virginica. The double matrix meas consists of four types of measurements on the flowers: the length and width of sepals and petals in centimeters, respectively.

Store the data in a table array.

t = table(species,meas(:,1),meas(:,2),meas(:,3),meas(:,4),...
'VariableNames',{'species','meas1','meas2','meas3','meas4'});
Meas = dataset([1 2 3 4]','VarNames',{'Measurements'});

Fit a repeated measures model, where the measurements are the responses and the species is the predictor variable.

rm = fitrm(t,'meas1-meas4~species','WithinDesign',Meas);

Perform a multiple comparison of the estimated marginal means of species.

tbl = multcompare(rm,'species')

tbl=6×7 table
      species_1         species_2       Difference     StdErr       pValue       Lower       Upper  
    ______________    ______________    __________    ________    __________    ________    ________

    {'setosa'    }    {'versicolor'}     -1.0375      0.060539    9.5606e-10     -1.1794    -0.89562
    {'setosa'    }    {'virginica' }     -1.7495      0.060539    9.5606e-10     -1.8914     -1.6076
    {'versicolor'}    {'setosa'    }      1.0375      0.060539    9.5606e-10     0.89562      1.1794
    {'versicolor'}    {'virginica' }      -0.712      0.060539    9.5606e-10    -0.85388    -0.57012
    {'virginica' }    {'setosa'    }      1.7495      0.060539    9.5606e-10      1.6076      1.8914
    {'virginica' }    {'versicolor'}       0.712      0.060539    9.5606e-10     0.57012     0.85388

The small $$p$$-values (in the pValue field) indicate that the estimated marginal means for the three species significantly differ from each other.

Perform Multiple Comparisons with Specified Options

Load the sample data.

load repeatedmeas

The table between includes the between-subject variables age, IQ, group, gender, and eight repeated measures y1 through y8 as responses. The table within includes the within-subject variables w1 and w2. This is simulated data.

Fit a repeated measures model, where the repeated measures y1 through y8 are the responses, and age, IQ, group, gender, and the group-gender interaction are the predictor variables. Also specify the within-subject design matrix.

R = fitrm(between,'y1-y8 ~ Group*Gender + Age + IQ','WithinDesign',within);

Perform a multiple comparison of the estimated marginal means based on the variable Group.

T = multcompare(R,'Group')

T=6×7 table
    Group_1    Group_2    Difference    StdErr     pValue       Lower      Upper 
    _______    _______    __________    ______    _________    _______    _______

       A          B         4.9875      5.6271      0.65436    -9.1482     19.123
       A          C         23.094      5.9261    0.0021493     8.2074     37.981
       B          A        -4.9875      5.6271      0.65436    -19.123     9.1482
       B          C         18.107      5.8223     0.013588     3.4805     32.732
       C          A        -23.094      5.9261    0.0021493    -37.981    -8.2074
       C          B        -18.107      5.8223     0.013588    -32.732    -3.4805

The small $$p$$-value of 0.0021493 indicates that there is significant difference between the marginal means of groups A and C. The $$p$$-value of 0.65436 indicates that the difference between the marginal means for groups A and B is not significantly different from 0.

multcompare uses the Tukey-Kramer test statistic by default. Change the comparison type to the Scheffe procedure.

T = multcompare(R,'Group','ComparisonType','Scheffe')

T=6×7 table
    Group_1    Group_2    Difference    StdErr     pValue       Lower      Upper 
    _______    _______    __________    ______    _________    _______    _______

       A          B         4.9875      5.6271      0.67981    -9.7795     19.755
       A          C         23.094      5.9261    0.0031072     7.5426     38.646
       B          A        -4.9875      5.6271      0.67981    -19.755     9.7795
       B          C         18.107      5.8223     0.018169     2.8273     33.386
       C          A        -23.094      5.9261    0.0031072    -38.646    -7.5426
       C          B        -18.107      5.8223     0.018169    -33.386    -2.8273

The Scheffe test produces larger $$p$$-values, but similar conclusions.

Perform multiple comparisons of estimated marginal means based on the variable Group for each gender separately.

T = multcompare(R,'Group','By','Gender')

T=12×8 table
    Gender    Group_1    Group_2    Difference    StdErr     pValue       Lower        Upper   
    ______    _______    _______    __________    ______    ________    _________    __________

    Female       A          B         4.1883      8.0177     0.86128      -15.953        24.329
    Female       A          C         24.565      8.2083    0.017697       3.9449        45.184
    Female       B          A        -4.1883      8.0177     0.86128      -24.329        15.953
    Female       B          C         20.376      8.1101    0.049957    0.0033459        40.749
    Female       C          A        -24.565      8.2083    0.017697      -45.184       -3.9449
    Female       C          B        -20.376      8.1101    0.049957      -40.749    -0.0033459
    Male         A          B         5.7868      7.9498     0.74977      -14.183        25.757
    Male         A          C         21.624      8.1829    0.038022       1.0676        42.179
    Male         B          A        -5.7868      7.9498     0.74977      -25.757        14.183
    Male         B          C         15.837      8.0511     0.14414      -4.3881        36.062
    Male         C          A        -21.624      8.1829    0.038022      -42.179       -1.0676
    Male         C          B        -15.837      8.0511     0.14414      -36.062        4.3881

The results indicate that the difference between marginal means for groups A and B is not significant from 0 for either gender (corresponding $$p$$-values are 0.86128 for females and 0.74977 for males). The difference between marginal means for groups A and C is significant for both genders (corresponding $$p$$-values are 0.017697 for females and 0.038022 for males). While the difference between marginal means for groups B and C is significantly different from 0 for females ($$p$$-value is 0.049957), it is not significantly different from 0 for males ($$p$$-value is 0.14414).

Input Arguments

rm — Repeated measures model

RepeatedMeasuresModel object

Repeated measures model, returned as a RepeatedMeasuresModel object.

For properties and methods of this object, see RepeatedMeasuresModel.

var — Variables for which to compute marginal means

character vector | string scalar

Variables for which to compute the marginal means, specified as a character vector or string scalar representing the name of a between- or within-subjects factor in rm. If var is a between-subjects factor, it must be categorical.

Data Types: char | string

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: tbl = multcompare(rm,var,'By',C)

Alpha — Significance level

0.05 (default) | scalar value in the range of 0 through 1

Significance level of the confidence intervals for population marginal means, specified as the comma-separated pair consisting of 'alpha' and a scalar value in the range of 0 through 1. The confidence level is 100*(1–alpha)%.

Example: 'alpha',0.01

Data Types: double | single

By — Factor to perform comparisons by

character vector | string scalar

Factor to do the comparisons by, specified as the comma-separated pair consisting of 'By' and a character vector or string scalar. The comparison between levels of var occurs separately for each value of the factor you specify.

If you have more than one between-subjects factors, A, B, and C, and if you want to do the comparisons of A levels separately for each level of C, then specify A as the var argument and specify C using the 'By' argument as follows.

Example: 'By',C

Data Types: char | string

ComparisonType — Type of critical value to use

'tukey-kramer' (default) | 'dunn-sidak' | 'bonferroni' | 'scheffe' | 'lsd'

Type of critical value to use, specified as the comma-separated pair consisting of 'ComparisonType' and one of the following.

Comparison Type	Definition
'tukey-kramer'	Default. Also called Tukey’s Honest Significant Difference procedure. It is based on the Studentized range distribution. According to the unproven Tukey-Kramer conjecture, it is also accurate for problems where the quantities being compared are correlated, as in analysis of covariance with unbalanced covariate values.
'dunn-sidak'	Use critical values from the t distribution, after an adjustment for multiple comparisons that was proposed by Dunn and proved accurate by Sidák. The critical value is $$\left|t\right|=\frac{\left|{\overline{y}}_i-{\overline{y}}_j\right|}{\sqrt{MSE\left(\frac{1}{n_i}+\frac{1}{n_j}\right)}}>t_{1-\eta /2,v,}$$ where $$\eta =1-{\left(1-\alpha \right)}^{{}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\left(\begin{array}{l}k\\ 2\end{array}\right)$}\right.}}$$ and ng is the number of groups (marginal means). This procedure is similar to, but less conservative than, the Bonferroni procedure.
'bonferroni'	Use critical values from the t distribution, after a Bonferroni adjustment to compensate for multiple comparisons. The critical value is $$t_{\alpha /2\left(\begin{array}{c}ng\\ 2\end{array}\right),v},$$ where ng is the number of groups (marginal means), and v is the error degrees of freedom. This procedure is conservative, but usually less so than the Scheffé procedure.
'scheffe'	Use critical values from Scheffé's S procedure, derived from the F distribution. The critical value is $$\sqrt{\left(ng-1\right)F_{\alpha ,ng-1,v}},$$ where ng is the number of groups (marginal means), and v is the error degrees of freedom. This procedure provides a simultaneous confidence level for comparisons of all linear combinations of the means, and it is conservative for comparisons of simple differences of pairs.
'lsd'	Least significant difference. This option uses plain t-tests. The critical value is $$t_{\alpha /2,v},$$ where v is the error degrees of freedom. It provides no protection against the multiple comparison problem.

Example: 'ComparisonType','dunn-sidak'

Output Arguments

tbl — Results of multiple comparison

table

Results of multiple comparisons of estimated marginal means, returned as a table. tbl has the following columns.

Column Name	Description
Difference	Estimated difference between the corresponding two marginal means
StdErr	Standard error of the estimated difference between the corresponding two marginal means
pValue	p-value for a test that the difference between the corresponding two marginal means is 0
Lower	Lower limit of simultaneous 95% confidence intervals for the true difference
Upper	Upper limit of simultaneous 95% confidence intervals for the true difference

References

[1] G. A. Milliken, and Johnson, D. E. Analysis of Messy Data. Volume I: Designed Experiments. New York, NY: Chapman & Hall, 1992.

Version History

Introduced in R2014a