File:T-test vs equivalence test.png

English: Chances to pass the t-test or the equivalence test depending on the actual error 𝜇. Different lines denote different sample sizes obtained from different a priori choices of the standard deviation σ. The golden, solid curve always represents a correctly estimated sample size, the green curve a sample which is too large and the other curves samples which are too small. Neglecting the t-test type II (or the equivalence test type I) error implies a power of 50%. The thick grey lines denote 𝛥=1%. For comparison: in the bottom graph, the incorrect decisions of a reference test are red, the correct decisions are coloured blue. The reason for red areas to exist are economic considerations to limit the test costs: further increasing the sample size towards infinity would make the red areas disappear, at least for the revised t-test or the equivalence test. For the t-test, the areas with systematic error 𝜇>1% and 𝜇<−1% remain blue, but the inner turns red. This behaviour is counterintuitive to the idea that the error of a statistic test goes to zero as the sample size goes to infinity.

Original Source: Figure 2 from https://doi.org/10.1007/s11116-019-09991-9

Reworked version source code see https://gist.github.com/xor2k/2256abe4106cc8ac04a24f42bd36f2c6