What a p-value means
The p-value is the probability of getting a test statistic at least as extreme as yours if the null hypothesis were true. A small p-value means your data would be surprising under H₀, which counts as evidence against it. It is not the probability that H₀ is true, and it says nothing about how large or important the effect is. The American Statistical Association’s 2016 statement on p-values makes exactly these points.
Choosing the tail
Use a two-tailed test when the alternative is “different from” (≠). Use a one-tailed test only when the alternative is directional (> or <) and was decided before seeing the data. Chi-square and F tests are almost always right-tailed, because large statistics indicate a poor fit or big differences.
Frequently asked questions
Is p = 0.05 significant?
By the usual convention, a result is significant when p < α. At exactly 0.05 with α = 0.05 it is borderline; report the exact p-value rather than just “significant”.
How do I get the degrees of freedom?
One-sample or paired t: n − 1. Two-sample pooled t: n₁ + n₂ − 2 (Welch’s test uses a formula; the t-test calculator computes it). Chi-square goodness of fit: categories − 1; independence: (rows − 1)(columns − 1).
Why is my two-tailed p-value double the one-tailed?
Because a two-tailed test counts extreme results in both directions, and the t and z curves are symmetric.