Repeated measures ANOVA

Definitions & Key takeaways

Analysis of variance, or simply, ANOVA, is a type of parametric test used to compare the means of multiple groups. It is used to determine if there's a significant difference between the means or averages of three or more groups. Repeated measures ANOVA help to analyze change among these means over different times, and it is widely used in fields such as psychology, medicine, and biology.

Watch video only
Analysis of variance, or simply, ANOVA, is a type of parametric statistical test used to determine if there’s a significant difference between the means or averages of three or more groups.
And significance is normally defined by a p-value of less than 0.05 or 5%. Now when doing any parametric test, there are three key assumptions that we have to make about the population.
First, the sample population must have been recruited randomly. Choosing names randomly ensures that the people included in the study will have similar characteristics to the target population.
This is important because that ensures that the results of the test can be applied to the target population - meaning it has good external validity!
The second assumption is that each individual in the sample was recruited independently from other individuals in the sample.
In other words, no individuals influenced whether or not any other individual was included in the study. For example, if two friends decided to get their blood pressures measured on the same day, and they were both included in the study, these two individuals would not be independent of each other and the second assumption would not be met.
Like random sampling, independent recruitment of individuals is important because it ensures that the sample population approximates the target population.
The third assumption is that the sample size is large enough to approximate the target population, which usually means having more than 20 people.
If it’s impossible to get a large sample size, then the sample population must follow a normal bell-shaped distribution for the characteristic being studied because that’s what we would expect to see in the target population.Okay, now let’s say there’s a certain blood pressure medication, called Medication A, and you want to figure out if it helps lower systolic blood pressure after taking it for three months and after taking it for six months.
So, you find 10 people and give each of them Medication A. Then, you measure each of their systolic blood pressures at time 1 - which is the time you initially gave them the medication - and then measure it again at time 2, let’s say 3 months later, and time 3, let’s say 6 months after they started taking the medication.
You find out that the mean systolic blood pressure measurement at time 1 is 138; at time 2 it’s 132, and at time 3 it’s 130.
Now, the next step is to figure out if 138, 132, and 130 are significantly different from one another, and you do that by performing an ANOVA test.
Specifically, we would use a repeated measures ANOVA test, because we’re looking at the same group of people at multiple time periods.
In a repeated ANOVA test, time is the independent variable, and in this example, systolic blood pressure is the dependent variable.
It might be tempting to think that medication type is the independent variable in this study, but this isn’t the case, since everyone in the study is taking the same medication type.The repeated measures ANOVA test is different than an independent one-way ANOVA test, which looks at multiple groups of people at one time point.
For example, let’s say there are three medications called Medication A, B, and C. A one-way ANOVA test would compare the systolic blood pressure measurements for people who have been taking one of the three types of medications for 6 months.
In this example, medication type is the independent variable instead of time, because you measure the blood pressure of every person in the study at the same time.
Typically, a repeated ANOVA test starts with two hypotheses. The first hypothesis is the null hypothesis, and it says that the means of each group are equal.
In other words, the null hypothesis is that the mean systolic blood pressure is the same for people at time 1, 2, and 3.
The second hypothesis is the alternate hypothesis, and it says that at least one group’s mean is significantly different from the others.
So, the alternate hypothesis in our example is that the mean systolic blood pressure is not the same for people at time 1, 2, and 3.
One important thing to know is that ANOVA doesn’t tell you which group’s mean is different than the others or whether the mean is higher or lower; it simply tells you that the groups’ means are not equal.
Now, there are six steps to test these hypotheses. The first step is to calculate the mean of each individual group and the overall mean or grand mean - which is the mean blood pressure measurements for all the groups.
Since the means for each group are 138, 132, and 130, we can calculate the overall mean by adding up each group - so 138 plus 132 plus 130, which is 400.
Then, we divide that by the number of groups, which is 3. So, the overall mean is 400 divided by 3, or approximately 133.
The second step is to find the between-group variation, which is also called the sum of squares-between, or the SSB. The sum of squares-between is a measure of how similar each group’s mean is to the overall mean.
To find the sum of squares-between, we start by subtracting each group’s mean from the overall mean and squaring it, which is called the squared difference.
Then, you multiply the squared difference by the number of people in that group. For a repeated ANOVA test, the number of people in each group stays consistent unless people drop out of the study in the middle of it.
In this example, there are 10 people in each group. So, for Time 1, we subtract the mean blood pressure of the Time 1 group, which is 138, from the overall mean, which is 133, and that equals negative 5.
The squared difference is negative 5 squared, or 25, and 25 times 10 is 250. For the Time 2 group, the mean is 132, so 133 minus 132 is 1, and 1 squared is still 1, and 1 times 10 equals 10.
For the Time 3 group, the mean is 130, so 133 minus 130 is 3. 3 squared is 9, and 9 times 10 is 90.
Now that we have the values for each group, we add them together to get the sum of squares-between. So, 10 plus 250 plus 90 is 350.
A larger sum of squares-between tells us that the group means and the overall mean are spread out or different from one another, and a smaller sum of squares-between tells us that the group means are fairly similar to the overall mean.
The third step in the ANOVA calculation is to find the within-group variation, which is also called the sum of squares-within, or SSW.
The sum of squares-within is a measure of how similar each individual blood pressure measurement is from its own group mean.
To find the sum of squares-within, you start by finding the squared differences for each person in one group, and to do this, you take each individual blood pressure measurement and subtract that group’s mean, then square it.
For example, let’s just take the first 3 systolic blood pressure measurements in the Time 1 group, which are 129, 142, and 143.
Since the group mean is 138, you subtract 138 from each individual measurement, so 129 minus 138 is negative 9, 142 minus 138 is 4, and 143 minus 138 is 5.
Then you square each number and add them all together to get the squared difference - so when you add up negative 9-squared, or 81, and 4-squared, or 16, and 5-squared, or 25, you get 122.
The squared difference is larger for groups with more people, so let’s say the squared difference of the Time 1 group is 330, and the squared differences for the Time 2 and Time 3 groups are 310 and 265.
As a general rule, if all of the groups have equal sample sizes - like if each group has 10 people - then groups with higher squared differences, like the Time 1 group, have more variation than groups that have lower squared differences, like the Time 3 group.
In other words, the individual blood pressure measurements for individuals in the Time 1 group are more spread out than the blood pressure measurements for individuals in the Time 3 group.
Now, to get the sum of squares-within, we add up all the squared differences for each group. So 330 plus 310 plus 265 equals 905.To do step 4, we have to know a little more about the sum of squares-within.
The sum of squares-within is made of subject-level variation, which is also called the sum of squares of subjects or SSs, and random error, which is also called the sum of squared error or SSE.
The values for the sum of squares of subjects and sum of squared-error add up to the value of the sum of squares-within.
The sum of squares of subjects is basically the variation caused by differences in people’s individual characteristics, like sex, age, or genetic differences.
For example, let’s say we’re measuring blood pressure in two groups of people. People who are older tend to have higher blood pressure, so if there are lots of older people in one group and lots of younger people in the other group, then the individual blood pressure measurements in the first group will be higher than the measurements in the second group, and the sum of squares of subjects will be high.
The cool thing about repeated measures ANOVA tests is that we don’t count the sum of squares of subjects, because people are the same in every group.
For example, if there are older people in the Time 1 group, there will also be older people in the Time 2 and Time 3 groups, because it’s the exact same people!
The other part of within-group variation is the sum of squared-error. The sum of squared-error is simply the result of uncontrolled factors in an experiment.
For example, blood pressure tends to be slightly higher on cold days compared to warm days. So, if we’re comparing the blood pressure in two groups of people, but the first group’s blood pressure was taken on a colder day and the second group’s was taken on a warmer day, it’s likely that the first group will have slightly higher blood pressure.
Oftentimes in an experiment, there will be many small unknown sources of error that contribute to individual differences between groups.Since sum of squares of subjects is redundant in a repeated measures ANOVA test, we only use the sum of squared-error.
We can calculate sum of squared-error by subtracting the sum of squares of subjects from the within-group variation, or sum of squares-within.
We already calculated the sum of squares-within, which is 905, so the fourth step is to figure out the sum of squares of subjects and the sum of squared-error.To calculate the sum of squares of subjects, you first find the mean blood pressure value of each subject for each of the three groups.
For example, let’s say the first person in the study had systolic blood pressure measurements of 129, 127, and 122 for Times 1, 2, and 3.
So, the mean blood pressure for that person would be 129 plus 127 plus 122, which is 378, divided by 3, which equals 126.
Then, to get the squared difference for that person, subtract the grand mean, which is 133, from 126, and square it. So 126 minus 133 is negative 7, and negative 7-squared is 49.
To get the sum of squares of subjects, you get the squared difference for each of the ten people in the study, and then add them up and multiply that number by the number of groups, which in this case is 3.
So let’s say the sum of all the squared differences is 290, and 290 times 3 is 870. Now that we have the sum of squares of subjects, which is 870, we can subtract it from the within-group variation, or sum of squares-within, which is 905, to get the sum of squared-error.
So, 905 minus 870 is 35. The fifth step in the repeated measures ANOVA is to find the mean square of the treatment, or MST, and mean square of the error, or MSE.
The mean square of the treatment, also known as mean square between, is another measure of the variability between groups, but it takes into account the number of groups being tested.
The mean square of the treatment is calculated by dividing the sum of squares-between by its degrees of freedom, which is the number of groups minus 1.
Since the sum of squares-between is 350 and there are 3 groups, the mean square of the treatment is 350 divided by 3 minus 1, or 2.
So 350 divided by 2 is 175. The mean square of the error is like the sum of squared-error, but it takes into account the number of people in the study and the number of groups in the study.
To calculate the mean square of the error, the sum of squared error is divided by the degrees of freedom of error, which in this case is the sample size in all the groups minus 1, times the number of groups in the study minus 1.
In this example, there are 10 people and 3 groups, so the degrees of freedom of error will be 10 minus 1, or 9, times 3 minus 1, or 2, which equals 18.
Then, the sum of squared-error is 35, so the mean square of the error is 35 divided by 18, or 1.94.The sixth and final step in the ANOVA test is to calculate the F-statistic or F-stat, which is the mean square of the treatment, or the explained variation, divided by the mean square of the error, or the unexplained variation.
In our example, the mean square of the treatment is 175 and the mean square of the error is 1.94, so 175 divided by 1.94 equals 90.2.
To figure out if this is a significant F-stat, we have to compare the F-stat to the critical value for the study, which is a predetermined number used to determine whether or not to reject the null hypothesis.
If the value of the F-stat is greater than the critical value, then the null hypothesis is false and we can conclude that there is a significant relationship between medication and blood pressure.Critical values can be found on an F-distribution table like this one, which has the degrees of freedom for the sum of squares-between on the top and the degrees of freedom for the sum of squares- within on the side.
In repeated measures ANOVA, the side column represents the degree of freedom of the sum of squared-error portion of the sum of squares-within, instead of the total value.
Typically, the significance level for the F-distribution table is 0.05, or 5%. Using 2 degrees of freedom for the sum of squares-between and 18 degrees of freedom for the sum of squared error, we find a critical value of 3.55.
The value of our F-stat is 90.2, which is much higher than the critical value of 3.55, so we can reject the null hypothesis and conclude that at least one group’s mean systolic blood pressure is significantly different from the others.ANOVA tests are most often calculated using statistical software, and the software will often provide a p-value.
This p-value is the probability of getting a given F-stat or a higher F-stat, if the null hypothesis is true. In short, the p-value cuts out the step of finding the critical value.
So, if we use a significance level of 0.05, then a test with a p-value of less than 0.05 will indicate that the null hypothesis is false and that at least one group’s mean systolic blood pressure is different from the others.Finally, it’s important to keep in mind that if the assumptions of an ANOVA test are not met, we can’t be sure that the results can be applied to the target population, so an ANOVA shouldn’t be used.
Instead, for a repeated measures ANOVA, we could use a non-parametric test called the Friedman test, which doesn’t rely on parametric assumptions.
In short, the Friedman test compares the medians of the groups to determine if one of the group’s medians is different than the others.
So, in this case, if the median blood pressure is different at one time point compared to the others. Parametric tests are generally favoured over non-parametric tests, so the results of the Friedman test are not considered as strong of evidence as the ANOVA test.Alright, as a quick recap, ANOVA tests are a type of parametric test used to compare the means of multiple groups.
Each ANOVA test has a null hypothesis and an alternate hypothesis, and to test these hypotheses we can calculate an F-statistic by dividing the mean square of the treatment by the mean square of the error.
This F-statistic can be compared to a critical value to determine if the means of the groups are equal or not. There are three assumptions, random sampling, independent recruitment, and large sample size or normal distribution, that must be met in order to complete an ANOVA test.
And if these assumptions are not met, the Friedman test can be used to determine if the medians of the groups are equal or not.
recap Anova tests are a type of parametric test used to compare the means of multiple groups Each Anova test has a null hypothesis and an alternate hypothesis And to test these hypotheses we can calculate an F statistic by dividing the mean square of the treatment by the mean square of the error This F statistic can be compared to a critical value to determine if the means of the groups are equal or not There are three assumptions random sampling independent recruitment and large sample size or normal distribution that must be met in order to complete an anova test And if these assumptions are not met then the Friedman test can be used to determine if the