T-Test Calculator
A t-test calculator determines whether the difference between group means is statistically significant. Enter your sample data or summary statistics, choose a one-sample, two-sample or paired test, and the calculator returns a t-statistic, p-value and 95% confidence interval. A p-value below your significance threshold (typically 0.05) means a difference this large is unlikely to be due to chance alone.
- ✓One-sample, two-sample and paired t-tests
- ✓Raw data or summary statistics input
- ✓P-value, confidence interval and Cohen's d
Your test results will appear here
Enter data on the left, or load an example, then click Run t-test
The confidence interval is two-sided at the confidence level that matches α.
| Group | n | Mean | SD | SE |
|---|
Which t-test should I use?
Start with how the data were collected, not with the numbers. A t-test is one of the core tools of inferential statistical analysis: it asks whether a gap between means is bigger than sampling noise would produce. The right version depends on what you are comparing.
| Your data | Groups | Question | Use this test |
|---|---|---|---|
| Sample + known population value | 1 | Does my sample differ from the expected value? | One-sample t-test |
| Two independent groups | 2 | Do these groups have different means? | Two-sample t-test |
| Same subjects measured twice | 2 (paired) | Did the measurement change? | Paired t-test |
| Two groups, different or unknown variances | 2 | Safer option when equal variances are not guaranteed | Welch's t-test (recommended default) |
| Three or more groups | 3+ | Multiple group comparison | Use ANOVA instead |
How to use the t-test calculator
- Pick the test type. One-sample compares a group against a fixed value. Two-sample compares separate groups. Paired compares two measurements on the same accounts, users or items.
- Choose raw data or summary statistics. Paste a column straight from a spreadsheet, or type the mean, standard deviation and n. If you have raw numbers but need those three values for a report, the descriptive statistics calculator produces them. The descriptive statistics guide explains what each one measures.
- Set the hypothesized value. For a one-sample test, that's your benchmark (μ₀). For the other tests, leave the hypothesized difference at 0 to test "no difference".
- Choose α and a direction. Pick a one-tailed test only if you committed to the direction before you saw the data.
- Click Run t-test. You get t, degrees of freedom, the p-value, the critical t, a confidence interval, Cohen's d, a plot of the t-distribution and an APA-style line ready to paste.
Worked example
Click A/B test in the calculator to load this one. A store split checkout traffic between a new design (Variant B, Sample 1) and the current one (Variant A, Sample 2), then recorded average order value in dollars for each session.
This is the result people misread most often. The p-value of 0.076 sits above 0.05, so the test does not reject the null hypothesis. That doesn't show the designs perform the same. The effect size is medium, and the confidence interval runs from a $0.26 loss to a $4.88 gain. With 12 to 14 sessions per variant, the test had roughly a 42% chance of detecting an effect this size. About 32 per variant would give 80% power.
Switch the test type to Student's and you get t(24) = 1.84, p = .079. The two versions barely differ here because the standard deviations are close. The calculator reports the Welch result as t(23.94) = 1.85, p = .076, d = 0.72.
How to interpret your results
The p-value
The p-value is the probability of getting a t-statistic at least as extreme as yours if the null hypothesis were true. It is not the probability that the null hypothesis is true. A small value means your data would be surprising if there were no real difference.
| p-value | Interpretation | Typical conclusion |
|---|---|---|
| < 0.001 | Extremely unlikely under H₀ | Strong evidence against the null |
| 0.001–0.01 | Very unlikely | Good evidence against the null |
| 0.01–0.05 | Unlikely | Conventional threshold for significance |
| 0.05–0.10 | Marginally unlikely | Borderline, depends on context |
| > 0.10 | Plausible under H₀ | Insufficient evidence to reject the null |
Effect size (Cohen's d)
Significance tells you whether a difference is likely real. It says nothing about whether it's big. Cohen's d expresses the difference in standard-deviation units so you can judge size on a common scale. A p-value of 0.03 with a d of 0.05 is a statistically significant but practically meaningless difference: the sample was large enough to pick up a tiny gap. Report both numbers.
| Cohen's d | Effect size | Example context |
|---|---|---|
| < 0.20 | Negligible | Barely detectable difference |
| 0.20–0.49 | Small | Visible in large samples |
| 0.50–0.79 | Medium | Meaningful in most applications |
| ≥ 0.80 | Large | Practically significant difference |
Cohen meant these cutoffs as rough guides. A d of 0.3 on revenue per account might be worth a lot of money. Reading p and d together is more useful than either alone:
| p-value | Cohen's d | What it usually means |
|---|---|---|
| < 0.05 | ≥ 0.80 | Significant and large. A difference worth acting on. |
| < 0.05 | 0.50–0.79 | Significant and medium. Worth investigating further. |
| < 0.05 | < 0.20 | Significant but tiny. Common with very large samples; check whether the gap matters. |
| ≥ 0.05 | ≥ 0.50 | Not significant, but the effect looks sizable. The sample is probably too small. |
| ≥ 0.05 | < 0.20 | Not significant and negligible. No meaningful difference detected. |
Confidence interval
The confidence interval is a range of plausible values for the true difference (or, in a one-sample test, the true mean). If the 95% interval excludes the hypothesized value, the two-tailed test at α = 0.05 is significant, and the reverse also holds. Width tells you precision. An interval from $5 to $7 pins the lift down; one from $0.20 to $12 says the effect is probably real but its size is still uncertain. If the group means and standard deviations in the results are unfamiliar territory, see how to read summary statistics.
T-test assumptions
Four conditions matter. The third is the one most often stated wrongly.
- Observations are independent. Each value comes from a different unit, and one value doesn't influence another. Several sessions from the same user in an A/B test break this, so aggregate to one value per user first. The paired test handles linked measurements by working on the differences.
- The outcome is measured on a continuous or interval scale, such as revenue, time on page or fill weight.
- The sampling distribution of the mean is approximately normal. This is about the distribution of the sample mean, not your raw values. With roughly 30 or more observations per group, the Central Limit Theorem usually makes the sample mean close to normal even when the raw data are skewed, like order values with a long right tail. With small samples, look at a histogram or box plot for heavy skew and extreme points. The guide on how to find outliers explains what to check, since a single extreme value can swing both the mean and the SD.
- The groups have equal variances. This applies to Student's t-test only. Welch's t-test removes the requirement.
T-test vs z-test vs ANOVA
| Method | When to use | Key requirement |
|---|---|---|
| T-test | Comparing one or two group means | Population SD unknown; works for small or large n |
| Z-test | Comparing means when σ is known | Known population standard deviation (rare in practice) |
| One-way ANOVA | Comparing three or more group means | Avoids the inflated false-positive rate of running many t-tests |
| Mann-Whitney U | Non-parametric alternative to the two-sample t-test | Very small n or extreme non-normality |
| Wilcoxon signed-rank | Non-parametric alternative to the paired t-test | Same, for paired data |
Why not run three t-tests instead of one ANOVA? Each test at α = 0.05 carries a 5% false-positive risk, so three of them push the chance of at least one false alarm to about 14%. The rank-based tests work on the order of values rather than the values themselves. It's the same idea behind Spearman's correlation, covered in Pearson vs Spearman.
Formulas
Every version divides the observed gap by its standard error. They differ in how SE and df are worked out.
| Test | Standard error | Degrees of freedom | Cohen's d |
|---|---|---|---|
| One-sample | s / √n | n − 1 | (x̄ − μ₀) / s |
| Welch's | √(s₁²/n₁ + s₂²/n₂) | Welch–Satterthwaite: (s₁²/n₁ + s₂²/n₂)² / [(s₁²/n₁)²/(n₁−1) + (s₂²/n₂)²/(n₂−1)] | (x̄₁ − x̄₂) / sp |
| Student's | sp √(1/n₁ + 1/n₂), where sp² = [(n₁−1)s₁² + (n₂−1)s₂²] / (n₁+n₂−2) | n₁ + n₂ − 2 | (x̄₁ − x̄₂) / sp |
| Paired | sd / √n (SD of the differences) | n − 1 (pairs) | d̄ / sd |
The confidence interval is the estimate ± critical t × SE. P-values come from the t-distribution's cumulative function, calculated in your browser with full precision. If you set a non-zero hypothesized difference, Cohen's d measures the distance from that value instead of from zero.
Check that a difference is real before you act on it.
Use the free calculator for a quick read on an A/B test, a before-and-after rollout, a benchmark comparison or a pair of KPIs from two segments. Research teams use the same tests for study results, and it works for educational research and coursework where an APA-formatted result is required. It tests one comparison at a time. It doesn't pick a study design or correct for multiple comparisons.
Frequently asked questions
Choosing a test, reading the output and reporting it.
What is the difference between a one-sample and two-sample t-test?+
A one-sample t-test compares a single sample mean to a known or hypothesized value, for example whether average session duration for a new feature cohort differs from a 3-minute site benchmark. A two-sample t-test compares the means of two independent groups, for example whether users on Plan A have longer sessions than users on Plan B.
When should I use a paired t-test vs an independent samples t-test?+
Use a paired t-test when the same subjects or units are measured twice, such as revenue per account before and after a product upgrade, or the same users tested on two interface designs. Use an independent samples t-test when the two groups are separate and no value in one group is linked to a specific value in the other.
What does a p-value of 0.05 mean in a t-test?+
A p-value of 0.05 means that if the null hypothesis were true (the groups really had the same mean), you would see a difference at least this large about 5% of the time. It does not mean there is a 95% probability that the alternative hypothesis is correct. Values below 0.05 are the conventional cutoff for statistical significance, but report Cohen's d next to the p-value so readers can judge whether the difference is big enough to matter.
Should I use Student's t-test or Welch's t-test?+
Use Welch's t-test by default for two independent samples. It does not assume the groups have equal variances, and when the variances are equal it gives almost the same answer as Student's t-test. Student's pooled-variance test depends on the equal-variance assumption, which you can rarely confirm in practice, so many statisticians recommend Welch's as the standard choice.
Can I run a t-test on summary statistics instead of raw data?+
Yes. If you have the mean, standard deviation and sample size (n) for each group, switch the calculator to Summary statistics and enter those values. This helps when you are working from a published report or a dashboard export, or when the raw data sits in another system. For a paired test, enter the mean and standard deviation of the differences and the number of pairs.
What is the minimum sample size for a t-test?+
The formula works with as few as 2 observations per group, but results from very small samples are unreliable. The size you need depends on the effect you want to detect. For 80% power at a two-tailed alpha of 0.05, a two-sample test needs about 64 per group to detect a medium effect (d = 0.5) and about 26 per group for a large effect (d = 0.8). If your study is smaller than that, a non-significant result may mean too little data rather than no real difference.
What is a one-tailed vs two-tailed t-test?+
A two-tailed test checks for a difference in either direction (Group A is not equal to Group B). A one-tailed test checks one direction only, for example whether Feature B converts better than Feature A rather than simply differently. Use a two-tailed test unless you set a directional hypothesis before collecting data. One-tailed tests have more power in the chosen direction, which is exactly why they should never be picked after seeing which result looks better.
How do I report t-test results?+
APA style gives the degrees of freedom in parentheses, then the t-statistic, the p-value and an effect size: t(48) = 2.34, p = .023, d = 0.66. APA drops the leading zero on p-values because they cannot exceed 1. Include the mean and standard deviation for each group, and add the confidence interval for the difference as 95% CI [lower, upper]. The calculator writes this line for you under the results.
Is my data sent to a server?+
No. Parsing and every calculation run in your browser. The tool has no share links and saves nothing.