File Exchange

image thumbnail

Reliability study designer

version (2.6 KB) by Joris Meurs
Calculation of the number of required subjects and replications in a reliability assessment


Updated 18 Apr 2016

View License

File name : 'Shoukri.m'. This file can be used
for calculating the required number of subjects
at a desired kappa value for reliability
Five input values : 'alfa', 'omega', 'pi', 'rho' and 'kappa'
Three output value : 'k', 'n' and 'N'
alfa : significance level (value between 0 and 1)
omega : desired interval of kappa (value between 0 and 1)
pi : probability of positive rating (value between 0 and 1)
rho : within-subject interclass correlation
kappa : desired coefficient of agreement (value between 0 and 1)

k : number of subjects (value larger than 1)
n : number of replication (value equal to or larger than 1)
N : number of allocations (value larger than 1)
Figure : plot of the number of subjects versus the number of replications
Input syntax : Shoukri(alfa,omega,pi,kappa)
NOTE: No default values are set
[1] : Shoukri, M. M. (2010). Measures of interobserver agreement and reliability: CRC Press.
[2] : Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174.
[3] : Shoukri, M., Asyali, M. & Walter, S. (2003). Issues of cost and efficiency in the design of reliability studies. biometrics. 59(4): p. 1107-1112.
[4] : Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement. 20(1): p. 37-46

Developed by Joris Meurs BASc (2015)

Cite As

Joris Meurs (2020). Reliability study designer (, MATLAB Central File Exchange. Retrieved .

Comments and Ratings (2)

Joris Meurs

Requirements are updated now

You need to list the Statistics&MachineLearning Toolbox as a requirement for this. "icdf" is not available in the base package.


Updated required products

Added picture

Change of title

1. Plot of number of subjects vs. number of replications added
2. Number of input and output arguments changed

Title name is changed

MATLAB Release Compatibility
Created with R2014b
Compatible with any release
Platform Compatibility
Windows macOS Linux