Borrar filtros
Borrar filtros

Effect size, statistical power of the test, and confidence interval (of hypothesis testing)

18 visualizaciones (últimos 30 días)
I am using the following two-sample tests for non-normal distributions:
  1. chi2gof
  2. kstest2
  3. ranksum
  4. kruskalwallis
All of them return a p-value, i.e. a p-value for chi2gof, a p-value for kstest2, a p-value for ranksum, and a p-value for kruskalwallis.
Since the p-value is not enough to understand the data/distributions (for example, please see Sullivan & Feinn (2012), Dunkler et al.(2020), Greenland (2016), and du Prel et al. (2009)), I would like to calculate the effect size, the statistical power of the test, and the confidence interval (of hypothesis testing).
I would use the following MATLAB functions:
  1. meanEffectSize (two-sample effect size computations)
  2. sampsizepwr (power of the test)
  3. not found anything about the confidence interval (except for the Two-sample t-test (ttest2) that I am not using)..
However:
  1. I am not sure if both functions, meanEffectSize and sampsizepwr, can be "compatible" with (i) non-normal distributions and (ii) with the 4 previously mentioned tests.
  2. According to the documentation I found on sampsizepwr, it looks like that the calculation of the statistical power of the test works only with normal distributions, but I am not sure.
  3. Those 4 mentioned tests do not provide the confidence interval. Maybe other MATLAB function do, but I was not able to find them.
I have then 2 questions:
Question 1. Is there anyone who could kindly enlighten me on the "compatibility" of meanEffectSize and sampsizepw with (i) non-normal distributions and (ii) with the 4 previously mentioned tests? In other words, can I use meanEffectSize and sampsizepw for non-normal distributions and for any of the 4 mentioned statistical tests? Or, meanEffectSize and sampsizepw can be used only in some cases / with some assumptions?
Question 2. Is there anyone who could kindly tell me if there are MATLAB functions to calculate the confidence intervals for the 4 mentioned statistical tests? (So far, I found only that the Two-sample t-test (ttest2) provides the confidence interval, although I am not using that tests)

Respuestas (1)

Gautam
Gautam el 21 de Ag. de 2024
Hello Sim,
The functions meanEffectSize can be used for the non-parametric tests that you’ve mentioned. By default, the function uses Cohen’s D for effect size measure, but you can set the “effect” argument of the function to “cliff” which uses the Cliff’s Delta method orktest Kolmogorov-Smirnov test statis both of which can be used for non- normal distributions. You can use bootstrapping to estimate the confidence intervals, which “meanEffectSize” supports.
However, the function “sampsizepwr supports tests for only normally distributed data as mentioned in the MathWorks documentation for the function. For the estimation of the statistical power, it’d be better to use bootstrapping methods.
There are no straightforward ways of calculating confidence interval for non-parametric tests. However, bootstrapping or other resampling methods could be used to approximate or derive confidence intervals for the tests you mentioned.
1. For Chi-Square Goodness-of-Fit Test, you can calculate confidence intervals for proportions if you are testing categorical data. Use the binofit function to get confidence intervals for proportions if applicable
2. For Two-Sample Kolmogorov-Smirnov Test bootstrapping can be used to approximate confidence intervals for the test statistic
3. For Wilcoxon Rank-Sum Test, again bootstrapping to estimate confidence intervals could be used. You can also consider using the bootci function to compute confidence intervals through resampling.
4. For Kruskal-Wallis Test, use bootstrapping to estimate confidence intervals for group medians or effect sizes. Use the multcompare function after kruskalwallis to perform multiple comparisons, which can help infer differences between groups.
Please refer to the following MathWorks documentation for more information of these functions.
1. binofitfor confidence intervals for proportions:
2. bootcito compute confidence intervals using bootstrapped samples
3. multcompare to perform multiple comparisons for inference differences between groups
Hope this helps.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by