How to Calculate the Effect Size at p=.05

Hello All,

Suppose I have caluclated the effect size for two different measurements (using the computeCohen_d(x1, x2, varargin) toolbox).

Now I want to determine the effect size, at which a measurement-difference becomes significant.

In other words: How to I calculate the effect size at p=.05?

Thanks in advance!

[Jeff Miller]

'independent' or 'paired' ?

[Johannes Reifenrath]

Thanks for asking.

Independent.

Answer 1

[Jeff Miller]

You can get an answer by computing a confidence interval for the Cohen's d. Specifically, at p=0.05, one end of the confidence interval for that d is right at 0 (well, epsilon above 0, I suppose). In the independent case, the half width (HW) of a 95% confidence interval for d is approximately the following (e.g., Bird, 2023, Eq. 2):

HW = tinv(0.975,n1+n2-2) * sqrt(1/n1 + 1/n2)

where n1 and n2 are the sizes of the two independent samples. So, plug in your sample sizes and the HW that comes out should be the d value that would be just significant at p=0.05.

Apparently this is only "approximately" the HW because the standard error of the d value depends on the true d value, but the references in Bird suggest that the approximation is pretty good unless the true d is greater than 1-1.5.