Opening the book…
Suppose the quality checks are satisfactory and the treatment group has greater average closure. Could ordinary sampling variation help explain the difference? A null hypothesis supplies a reference claim, such as no difference between population means. A statistical test compares the observed data with that claim. The p value is the probability, assuming the null hypothesis and the statistical model are correct, of obtaining a test statistic at least as extreme as the one observed.
A p value is not the probability that the null hypothesis is true, and it does not measure the size or importance of an effect. A small p value cannot establish a specific biological mechanism. Report the effect estimate, uncertainty, design, and quality evidence alongside any p value.
An effect size describes the magnitude of a difference or association. In a scratch study, a treatment mean of 65% closure compared with a control mean of 50% gives a difference of 15 percentage points. The relative increase in closure is (65 − 50) ÷ 50 × 100 = 30%. These are different descriptions of the same comparison, so identify the scale used.
The study design helps determine the test. A two-sample t test can compare means from independent groups when its assumptions are suitable. Welch’s t test allows the groups to have unequal variances. A paired t test instead analyzes differences within pairs established by the design. The relevant distributional assumptions apply to the group data or the paired differences, respectively. Neither test turns related subsamples into independent observations.
For example, treatment and control results from the same donor may be paired. The relevant values are each donor’s treatment-minus-control difference, rather than two unrelated lists. If measurements are followed over time in the same well, they are also related; an analysis that assumes independent observations at every time point is unsuitable.
Comparing several concentrations creates more than one possible comparison. Multiple testing increases the opportunity to find a small p value by chance when many claims are examined. Use a planned analysis that handles the comparison structure, such as an appropriate overall model and specified follow-up comparisons. Do not repeatedly try different tests until one produces the desired result.
A result that does not meet a chosen significance threshold does not prove that the groups are identical. It may reflect little effect, high variation, limited sample size, or an unsuitable analysis. Equivalence is a separate question requiring a meaningful predefined margin and an appropriate design and analysis. “No detected difference” and “equivalent” are not interchangeable conclusions.
Inspect the data for errors, unexpected patterns, and whether the planned model is suitable. Counts, proportions, and strongly skewed values may need approaches beyond a simple mean comparison. A normality check with very few values cannot establish that all assumptions are satisfied. Document justified changes to the plan, and seek statistical guidance when needed.
Attribution: Images AI-generated using ChatGPT Modified by Emalee Mackenzie, Irvine Valley College. Licensed under CC BY-NC-SA
This page titled 15.10: Comparing Groups and Interpreting p Values was last modified on Wed, 21 Oct 2026 01:35:52 GMT and is shared under a CC BY-NC-SA 4.0 license and was authored, remixed, and/or curated by Emalee MacKenzie.
The book could not be loaded just now.