Two-sample t-test

Definitions & Key takeaways

The two-sample t-test is a statistical method used to compare the means of two independent samples of continuous data. It is used to determine whether the means of the two samples are significantly different from each other, taking into account the variability of the data within each sample.

The t-test calculates a t-value, which is then compared to a critical value from a t-distribution with degrees of freedom equal to the sum of the sample sizes minus two. If the calculated t-value is greater than the critical value, the null hypothesis of no difference between the means is rejected, indicating that there is a significant difference between the means of the two samples. The two-sample t-test is commonly used in a variety of research fields, including medicine, psychology, and engineering, to compare two groups or treatments.

Watch video only
The Student’s t-test or simply, the t-test, is a type of parametric statistical test used to determine if there’s a significant difference between the means or averages of two groups.
And significance is normally defined by a p-value of less than 0.05 or 5%. Now when doing any parametric test, there are three key assumptions that we have to make about the population.
First, the sample population must have been recruited randomly. Choosing names randomly ensures that the people included in the study will have similar characteristics to the target population.
This is important because that ensures that the results of the t-test can be applied to the target population - meaning it has good external validity!
The second assumption is that each individual in the sample was recruited independently from other individuals in the sample.
In other words, no individuals influenced whether or not any other individual was included in the study. For example, if two friends decided to get their blood pressures measured on the same day, and they were both included in the study, these two individuals would not be independent of each other and the second assumption would not be met.
Like random sampling, independent recruitment of individuals is important because it ensures that the sample population approximates the target population.
The third assumption is that the sample size is large enough to approximate the target population, which usually means having more than 20 people.
If it’s impossible to get a large sample size, then the sample population must follow a normal bell-shaped distribution for the characteristic being studied because that’s what we would expect to see in the target population.
Okay, now let’s say you want to figure out if a certain medication lowers systolic blood pressure. So you find 25 people who have been on the medication for 6 weeks, and figure out that the mean systolic blood pressure for the whole group is 130 mmHg.
Then, you find another 30 people who have not been taking that medication, and find out the mean systolic blood pressure for that group is 138 mmHg.
Now, to figure out if a difference in systolic blood pressure from 130 to 138 is significant, we could perform a t-test.
Specifically, since the two means were measured in two different populations, we would use an unpaired or two-sample t-test.
This is different than a paired t-test, which is used to compare the same population before and after the treatment. For example, a paired t-test could compare the systolic blood pressure measurements of a group of 25 people before using the medication to the systolic blood pressure measurements of the same people after using the medication for six weeks.
Typically, an unpaired t-test starts with two hypotheses. The first hypothesis is the null hypothesis, and it basically says that the difference in means between the two groups is equal to zero.
In other words, the null hypothesis is that taking the medication results in no difference in systolic blood pressure. The second hypothesis is the alternate hypothesis, and since a t-test can be either one-sided or two-sided, there are two versions of the alternative hypothesis.
The alternate hypothesis for a one-sided t-test would either state that the difference in means is a positive number or that the difference in means is a negative number.
The alternate hypothesis for a two-sided t-test would state that the difference in means for both groups is not equal to zero, but it wouldn’t specify if it was positive or negative.
Typically, researchers choose to use two-sided t-tests, since they usually don’t know how the medication will affect people who take it.
So, the two-sided alternative hypothesis for our study would state that the difference in means in systolic blood pressure for people that take the medication compared to people who don’t take the medication is not equal to zero.
To test these hypotheses, we need to calculate a t-score, which is a ratio of the difference in means between the two groups to the standard error of the difference in means between the two groups.
Now, for an unpaired t-test, the t-score depends on whether the variance between the two groups is equal or unequal. Variance is a measure of how spread out each individual blood pressure reading is from the group mean.
A large variance means that the numbers are very spread out from the mean, like if the mean blood pressure was 130 and the individual measurements included numbers like 112, 142, and 155.
A small variance means that the numbers are very close to the mean, like if the mean blood pressure was 130 and the individual measurements included numbers like 129, 131, and 135.
The variance is the square of the standard deviation - which is another measure of how spread out individual values are from the group mean.
As a general rule, if one group has a variance that’s more than double the other group’s variance, then the variance is unequal.
So, if the standard deviation for the no medication group is 12, then the variance would be 12 squared, or 144. And if the standard deviation for the no medication group is 5, then the variance would be 5 squared, or 25.
Since the variance of the no medication group is more than double the variance of the medication group, the variance is unequal.
Now to calculate the t-score, let’s start with the first part - the difference in means between the two groups. In our case, that’s the difference in the mean systolic blood pressure for individuals who took the medication and who didn’t take the medication, and it’s represented by the symbol d-bar.
The mean systolic blood pressure for the no medication group is 138 and the mean systolic blood pressure for the medication group is 130, so the difference in means is 8.
Alright, on to the second part - the standard error of the difference in means between the two groups, or simply, the standard error.
Put differently, the standard error tells us on average, how far the difference of means from the sample population - which we just calculated - is from the difference of means from the true population - which is usually just the general population.
Calculating the standard error for groups with unequal variance has a few steps. First, you find the variance for the first group, 144, and divide it by the sample size for the first group, which is 30, and that equals 4.8.
Then, you take the variance for the second group, 25, and divide it by the sample size for the second group, which is 25 as well, so that equals 1.
Those two numbers are then added together - so 4.8 plus 1, which equals 5.8 - and finally, you take the square root of that whole thing.
The square root of 5.8 is 2.4, so the standard error is 2.4. When the standard error is small like this, that means that the sample population is more representative of the general population.
To find the t-score, we divide the difference in means between the two groups, 8, by the standard error, 2.4, which equals 3.3.
To figure out if this is a significant t-score, we have to compare the t-score to the critical value for the study, which is a predetermined number used to determine whether or not to reject the null hypothesis.
If the absolute value of the t-score is greater than the critical value, then the null hypothesis is false and we can conclude that there is a significant relationship between medication and blood pressure.
Critical values can be found on a t-score table like this one, which has degrees of freedom on the side and significance level on the top.
The calculation for figuring out the degrees of freedom for an unpaired t-test with unequal variance is pretty complicated, so to save time we can just use the sample size from the group with the smaller population minus 1.
So, we’ll use the sample size from the medication group, which is 25 people, and 25 minus 1 equals 24 degrees of freedom.
The significance level is the p-value that’s determined by the researchers at the beginning of a study, and usually it’s just 0.05 or 5%.
Using 24 degrees of freedom and a significance level of 0.05, we find a critical value of 2.064. The absolute value of our t-score is 3.3, which is above the critical value of 2.064, so we can reject the null hypothesis and we can conclude that there is a relationship between medication and blood pressure.
Now, let’s switch gears and talk about how the calculation changes if the groups have equal variance. If the two groups have exactly the same variance, then each of the variances divided by the sample size would be the same.
But, since it’s pretty rare for two groups to have exactly the same variance, instead we use a pooled variance, which is a weighted average of the two variances.
So let’s say that the variance of the no medication group is 144 and the variance of the medication group is 121. First we figure out the sample size of the no medication group minus 1, so 30 minus 1, or 29.
We multiply it by the variance, 144, and that results in 4,176. Then we multiply the the sample size of the medication group minus 1, so 25 minus 1, or 24, by the variance, 121, which equals 2,904.
We add the two numbers up - so 4,176 plus 2,904 equals 7,080. Then, we divide 7,080 by the sum of sample size of the no medication group minus 1, or 29, and the sample size of the medication group minus 1, or 24, which equals 53.
So 7,080 divided by 53 is roughly 134. After that, the rest of the t-score calculation is pretty similar to the one we used for groups with unequal variance.
To calculate the standard error, we divide the pooled variance, 134, by the sample size of the no medication group, 30, which equals 4.5.
Then we divide the pooled variance by the sample size of the medication group, 25, and that equals 5.4. Then we add them together and take the square root - so 4.5 plus 5.4 is 9.9, and the square root of 9.9 is around 3.1.
That’s the standard error. To find the t-score, we divide the difference in means between the two groups, 8, by the standard error, 3.1, which equals 2.58.
Again, we have to compare the t-score to the critical value found on the t-chart. For groups with equal variance, the degrees of freedom is calculated by adding the two sample sizes together and subtracting 2.
So 25 plus 30 is 55, minus 2 is 53. Since 53 is between 40 and 60, we’ll use the significance value for the lower number on the t-chart, which is 40.
Using a significance level of 0.05, we find a critical value of 2.021. And since our t-score is 2.58, which is above the critical value of 2.021, we can reject the null hypothesis and conclude that there is a relationship between medication and blood pressure.
T-tests are most often calculated using statistical software, and the software will often provide a p-value. This p-value is the probability of obtaining a given t-score or a higher t-score, if the null hypothesis is true.
In short, the p-value cuts out the step of finding the critical value. So, if we use a significance level of 0.05, then a test with a p-value of less than 0.05 will indicate that the null hypothesis is false and that there is a significant relationship between medication and blood pressure.
Finally, it’s important to keep in mind that if the assumptions of an unpaired t-test are not met, we can’t be sure that the results of a t-test can be applied to the target population, so a t-test shouldn’t be used.
Instead, we could use a non-parametric test called the Wilcoxon rank-sum test, which doesn’t rely on parametric assumptions.
In short, the Wilcoxon rank-sum test compares the differences in medians of the two groups to determine if two variables are related, so in this case, if medication affects blood pressure.
Parametric tests are generally favoured over non-parametric tests, so the results from the Wilcoxon rank-sum test are not considered as strong of evidence for a relationship between two variables compared to the t-test.
Alright, as a quick recap, unpaired t-tests are a type of parametric test used to compare two groups of individuals. Each t-test has a null hypothesis and an alternate hypothesis, and to test these hypotheses we can calculate a t-score by dividing the difference in means between groups to the standard error of the difference in means.
The t-score calculation changes slightly, depending on if the two groups have unequal or equal variance, and the calculation for equal variance uses a pooled variance.
There are three assumptions, random sampling, independent recruitment, and large sample size or normal distribution, that must be met in order to complete a t-test.
And if these assumptions are not met, the Wilcoxon rank-sum test can be used to determine if there’s a relationship between the two variables.