Lower bound on p-value returned by corr

I'm using corr to calculate the p-value for the correlation between two datasets. It keeps returning a p-value of Zero.

Does anyone know the lower-bound on this calculation? I'm guessing its 1E-250 since that's the smallest I can generate with minor perturbations to the data.

In case it matters I'm using 2011A.

[Pamela Reinagel]

We cannot report p< if we don't kow what smallest reportable p value is below which Matlab sets p=0. Moreoever some journals now require reporting exact p values and prohibit p< statements. Is there any documentation as to what the value of p is, less than which Matlab's Spearman correlation returns p=0? Or is there any way to set alpha for a Spearman correlation or force it to return the exact p value? Or is it in the nature of the mathematics that the Spearman has a finite lowest measurable p value?

[the cyclist]

This is trickier to answer than I expected, and I don't think there is a single number that can be quoted in response.

The main reason it is tricky is that MATLAB uses a large-sample approximation (assuming your sample size is not small enough to do exactly), and how accurate this approximation will be -- and how precisely it will be reported -- is dependent on the sample size.

I figured this out via some sleuthing using the debugger, and stepping into corr and its subfunctions. Eventually, one gets to the lines where it calculates "Algorithm AS 89", as referenced at the bottom of the documentation page for corr (ref. 4).

I can think of a couple courses of action:

1. Try tweaking your data "closer" to each other until you get a non-zero value. It will likely be very small. Then I believe you can accurately state that your p-value is smaller than that.
2. Trying extracting your data and doing this in some other statistical software (e.g. R or SAS) and see if it reports something different from zero. (If you upload your data in a MAT file, I could try it in R.)

Answer 1

[the cyclist]

I don't know for sure, but my guess is that if you type "realmin" at the command prompt, that is the minimum you could get, theoretically. (For me, that's about 2.2e-308.)

Also, bear in mind that tiny p-values are generally not reliably accurate. That is one reason that small p-values are typically reported as, say, p < 0.001, rather than with the value itself.

[Will Dampier]

Thanks ... I also tend to report them as p < alpha. I just wanted to know the lower bound incase someone else actually asked.

[the cyclist]

Well, that is a subtly different thing. Ideally, as you probably know, the value of alpha (the defined threshold of statistical significance) should be set ahead of the experiment. That might be alpha=0.05. But you still quote the observed value of p. That value of p might be 0.036, and you could report p=0.036, as well as p < alpha. But if p is very tiny, like 0.000000036, then typically one reports p < 0.001, rather than p = ..., because such a tiny p-value is not actually accurate, even to within an order-of-magnitude.