Skip to content

DA5. Power Calculations

Statement

A power analysis is a calculation used to estimate the smallest sample size needed for an experiment, given a required significance level, statistical power, and effect size. It helps to determine if a result from an experiment or survey is due to chance, or if it is genuine and significant.

Think of an experiment or a survey that you have conducted or will conduct in your field of interest and use it to discuss the following questions.

Answer

We will taking problem 7.33 from the OpenIntro Statistics textbook as an example. The problem is as follows:

“7.33 Increasing corn yield. A large farm wants to try out a new type of fertilizer to evaluate whether it will improve the farm’s corn production. The land is broken into plots that produce an average of 1,215 pounds of corn with a standard deviation of 94 pounds per plot. The owner is interested in detecting any average difference of at least 40 pounds per plot. How many plots of land would be needed for the experiment if the desired power level is 90%? Assume each plot of land gets treated with either the current fertilizer or the new fertilizer” (Diez, Barr, & Çetinkaya-Rundel, 2019, p.284).

1. Describe the importance of power analysis in an experiment.

“Power analysis is important in an experiment because it helps to determine the smallest sample size needed to detect a significant effect size” (Diez, Barr, & Çetinkaya-Rundel, 2019). Power is the probability of correctly rejecting the null hypothesis when the alternative hypothesis is true. It is 1-β, where β is the probability of a Type 2 error, thus, higher power reduces the probability of Type 2 errors (Çetinkaya-Rundel, 2018).

In the context of the corn yield experiment, utilizing less than required plots of land render the experiment useless, while using more than necessary plots of land is a waste of resources. Especially in such expensive and time-consuming experiments, power analysis is crucial to utilize just the right amount of resources to detect a significant effect.

The four components of a power analysis are (Spotfire, 2024):

  • Effect size: The difference between the means of the two groups, which is 40 pounds in this case.
  • Sample size: The number of plots of land needed for the experiment, which is unknown.
  • Significance level (α): The probability of rejecting the null hypothesis when it is true, which is 0.05 in this case.
  • Power level (1-β): The probability of accepting the alternative hypothesis, which is 0.90 in this case.

A power analysis estimates one of these four parameters, when given the values for the remaining three.

2. How will you determine the appropriate significance level for a power analysis?.

The significance level α is the probability of rejecting the null hypothesis when it is true. It is usually set at 0.05, which means a room for 5% chance for a Type 1 error. After getting the p-value we compare it to α, and if α <= p-value, we reject the null hypothesis. If α > p-value, we fail to reject the null hypothesis (Cohen, 1988, p.4).

The other significance level is β which is the probability of accepting the null hypothesis when it is false. It is usually the complement of the power level; and since the power level is usually set at 0.80, β is 0.20, which means a room for 20% chance for a Type 2 error (Cohen, 1988, p.5).

3. Explain, in your own words, how you will conduct a power analysis in your experiment or survey.

The power analysis for the corn yield experiment expects us to find the sample size needed to detect an average difference of at least 40 pounds per plot with a power level of 90%.

  • Effect Size (EF) = 40 pounds.
  • For the significance level (α) = 0.05, the rejection region bounds are \(\(\pm 1.96 \times SE\)\).
  • For the power level (1-β) = 0.90, the rejection region bounds are \(\(1.28 \times SE\)\)
  • The standard error (SE) is calculated as \(\(\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}\)\)
  • The total rejection region bounds are \(\(1.96 \times SE + 1.28 \times SE = 3.24 SE\)\)
  • Then we do the analysis:
\[ \begin{align*} 3.24 \times SE &= 40 \\ SE &= \frac{40}{3.24} \\ \sqrt{\frac{94^2}{n} + \frac{94^2}{n}} &= 12.345 \\ \sqrt{\frac{17672}{n}}&= 12.345 \\ \frac{17672}{n} &= 152.415 \\ n &= \frac{17672}{152.415} \\ n &= 115.9 \end{align*} \]

Therefore, the farm needs 116 plots of land to detect an average difference of at least 40 pounds per plot with a power level of 90%.

References

‌