Correlation

Definitions & Key takeaways

A correlation is a statistical measure that quantifies the degree of relationship between two variables. In other words, it shows how closely two things are related. A correlation coefficient can range from -1 to 1, with a value of 1 indicating a perfect positive correlation and a value of -1 indicating a perfect negative correlation.

Watch video only
Correlation is a statistical technique that shows whether two quantitative variables are related, and also how strongly they’re related.
For example, let’s say you want to figure out if drinking more soda is correlated with having a higher body mass index or BMI.
So, you ask 100 people how many sugary beverages they drink in a week and then check each person’s height and weight to calculate their BMI.
You could plot these measurements or data points on a scatterplot, with the number of beverages on the x-axis and BMI on the y-axis, and where each data point represents one individual.
Typically, a trendline is drawn to best represent the pattern of data points on the plot, with roughly half the points above the line and half the points below the line.
Now, a positive correlation means that BMI increases as the number of beverages increases, and, if the two variables have a perfect positive correlation, then the trendline will pass through every single data point.
Now imagine that there’s a negative correlation. That means that the BMI decreases as the number of beverages increases, and, with a perfect negative correlation, the trendline also passes through every data point.
Finally, if there’s no correlation, then the data points will be randomly spread out all over the scatterplot, and the trendline will be flat with no positive or negative direction.
To figure out how strongly two variables are correlated, we can use the Pearson’s correlation test, which is a parametric test that measures how close or spread out the data points are from the trendline.
The Pearson’s correlation test calculates a correlation coefficient, which is a number that represents how well the two variables are correlated, and usually it’s written with a lowercase r.
The correlation coefficient can range from negative 1 - which represents perfect negative correlation - to positive 1 - which represents perfect positive correlation.
A correlation coefficient of 0 means that there’s no correlation between the two variables. Now, there are four key assumptions that are used in a Pearson’s correlation test, and an acronym for these assumptions is L-I-N-E, or LINE.
First, the relationship between the two variables has to be Linear, which means that the trendline drawn to represent the data points is a straight line.
Relationships that have a different type of curve, like in an exponential relationship or a U-shaped relationship, will have a low correlation coefficient because a straight trendline doesn’t match the shape of the data points.
The second assumption is that each individual in the sample was recruited Independently from other individuals in the sample.
In other words, no individuals influenced whether or not any other individual was included in the study. For example, if one person agrees to be in the study only if their friend can also be included in the study, then these two individuals would not be independent of each other and the second assumption would not be met.
Independent recruitment ensures that the people included in the study have similar characteristics to the target population, and that the results of the test can be applied to the target population - meaning it has good external validity!
Additionally, the sample population must have been recruited randomly, like if you chose 100 names randomly from a list of all names in the target population.
Like independent recruitment, random sampling is important because it ensures that the sample population approximates the target population.
The third assumption is that the errors between the observed and predicted values of y are Normally distributed around a value of x.
That might sound confusing - no worries. Let’s break it down, by reviewing the trendline, which shows a predicted y-value for a specific x-value.
For example, let’s say we have a trendline that looks like this. According to our trendline, for people that drink 2 sugary beverages per week, we predict that their BMI will be around 23.
But in reality, some people that drink 2 beverages will have a BMI that’s higher or lower than 23, simply because of individual differences, like how old a person is or how much a person exercises.
The distance a point is from the prediction line is called error, and points that are far from the line are said to have a high error.
Okay, so the third assumption says that the errors for a given point will follow a normally distributed bell curve, meaning most people will have low error, so their BMI will be clustered near the trendline; and fewer people will have high error, so their BMI will be higher or lower than the trendline.
And this is true for every value of x, or for each number of sugary beverages per week. The fourth assumption is that the data points must have Equal variance, which is also called homoscedasticity.
Basically, this means that the data points are equally spread out from the trendline for every value of x. If the data points are closer to the trendline on one end and further away on the other end, then the assumption of equal variance is not met and the data are heteroscedastic.
So going back to the Pearson’s correlation test, it starts with two hypotheses. The null hypothesis states that r equals zero, or in other words, there is no correlation between two variables.
And the alternate hypothesis is that r does not equal zero, or that there is a correlation between two variables. To test these hypotheses, we have to calculate the correlation coefficient, and there are five steps for doing this.
The first step is to find the sum of each of the x-values, and in this example, that’s the number of beverages. As a simple example, we’ll use a sample size of 5 people, and let’s say the number of beverages each person consumes is 3, 5, 8, 10, and 12.
Oftentimes, the sum is written with the Greek letter sigma, so the sum of all the x-values is 3 plus 5 plus 8 plus 10 plus 12 equals 38.
So our sample population of 5 people, drinks around 38 sugary beverages per week. The second step is to find the sum of each of the y-values, and in this example, that’s the BMI for each person.
So, let’s say the five individuals had BMI of 24, 23, 28, 29, and 35, and when we add them all together we get 139. This means that our sample population of 5 people has a combined BMI of 139.
The third step is to find the sum of each of the squared values of x and the sum of each of the squared values of y. So, when we square each x-value, we get 9, 25, 64, 100, and 144, and when we add them all together we get 342.
When we square each y-value, we get 576, 529, 784, 841, and 1225, and when we add them all together we get 3995. The fourth step is to find the sum of x times y.
To do this, we multiply the value of x for the first person by the value of y for the first person, so 3 times 24 is 72.
Then, we repeat that for the other four people - so, 5 times 23 is 115, 8 times 28 is 224, 10 times 29 is 290, and 12 times 35 is 420.
Finally, we add those values together to get 1121. The fifth step is to plug all the sums we just found into the equation for the correlation coefficient, which looks like this.
This equation has a lot of different parts, so let’s break it down. In the numerator, we have n, or the sample size, times the sum of x-times-y, minus the sum-of-x times the sum-of-y.
Since the sample size in our example is 5, we multiply 5 by the sum of x-times-y, which is 1121, and that equals 5605. Then, we multiply the sum-of-x, or 38, by the sum-of-y, or 139, which equals 5282.
So, the value of the numerator is 5605 minus 5282, which equals 323. In the denominator we have three different parts.
First, we multiply n by the sum of x-squared, and then subtract the sum-of-x, squared, from it. So, we use the same sample size, which is 5, times the sum of x-squared, which is 342, and that equals 1710.
Then, we’ll use the sum-of-x, which is 38, and square it, so 38-squared is 1444. To finish off that part, we’ll subtract the second number, 1444, from the first number, 1710, to get 266.
Okay, now we do the same thing for y for the second part. So, n - or 5 - times the sum of y-squared - or 3955 - equals 19,775.
And, the sum-of-y, or 139, squared is 19,321. So, 19,775 minus 19,321 equals 454.
For the third part, we multiply the two numbers we just got and find the square root of them. So, 266 times 454 is 120,764, and the square root of that is 347.5.
Great, so now we can divide the numerator, which is 323, by the denominator, which is 347.5, to get 0.93, which is the correlation coefficient.
As a general rule, correlation coefficients that are higher than 0.7 are considered strong, so since our correlation coefficient is higher than 0.7, we can be pretty sure that there’s a strong correlation between the number of sugary beverages consumed and BMI.
And, if we look at the scatterplot and trendline for these data, we can see that the trendline also matches up pretty well with the data points.
To figure out if the correlation is statistically significant, we have to compare the correlation coefficient that we calculated to a critical value, which is a predetermined number used to determine whether or not to reject the null hypothesis.
If the value of the correlation coefficient is greater than the critical value, then the null hypothesis is false and we can conclude that there is a correlation between the number of sugary beverages consumed and BMI.
Critical values can be found on an Pearson’s correlation table like this one, which has degrees of freedom on the side and significance level on the top.
The degrees of freedom (df) for a Pearson’s correlation test is the sample size minus 2. So, in our example, the degrees of freedom would be 5 minus 2, which equals 3.
The significance level is the p-value that’s determined by the researchers at the beginning of a study, and usually it’s just 0.05 or 5%.
Using 3 degrees of freedom and a significance level of 0.05, we find a critical value of 0.878. The value of our correlation coefficient is 0.93, which is above the critical value, so we can reject the null hypothesis and conclude that there is a correlation between the number of sugary beverages consumed and BMI.
Pearson’s correlation tests are most often calculated using statistical software, and the software will often provide a p-value.
This p-value is the probability of obtaining a given correlation coefficient or a higher coefficient, if the null hypothesis is true.
In short, the p-value cuts out the step of finding the critical value. So, if we use a significance level of 0.05, then a test with a p-value of less than 0.05 will indicate that the null hypothesis is false and that there is a correlation between the number of sugary beverages consumed and BMI.
Typically, the software will also provide the coefficient of determination, or the R-squared value, which is simply the correlation coefficient, squared.
So, the R-squared value for our example would be 0.93-squared, or 0.86. The R-squared value tells you how much variation in the y-variable is explained by the x-variable, as opposed to natural variation.
Said differently, can a person’s BMI be explained by the number of sugary beverages they drink, or is it more the result of other factors, like sex, age, or the amount they exercise.
In this example, an R-squared value of 0.86 means that 86% of the variation in BMI can be explained by the consumption of sugary beverages, and the remaining 14% is explained by other factors.
A high R-squared value like this one tells us that most of the variation we see between people with a high BMI and a low BMI is the result of drinking different amounts of sugary beverages, and specifically, that people with lower BMI drink fewer sugary beverages than people with a high BMI.
One important thing to keep in mind is that correlation is not the same as causation, so even though people with higher BMI drink more sugary beverages, it doesn’t mean that drinking sugary beverages actually causes a person to have a higher BMI.
Sometimes, the x-variable is actually a proxy or substitute measure for a different variable. For example, people that drink more sugary beverages likely have poorer diets than people that drink fewer sugary beverages, and diet is a known risk factor of high BMI.
So, it’s actually diet that’s causing an increase in BMI, not sugary beverages. A variable - like diet - that’s hidden by another variable - like the number of sugary beverages - is called a confounder.
Now, there are two key limitations when calculating correlation. The first limitation is that correlation is sensitive to outliers, or data points that are much higher or much lower than all the other data points.
For example, let’s say we add a sixth person to our example, and that person drinks 8 sugary beverages per week but has a very low BMI of 16.
In this case, the trendline would try to account for this new data point, but since it’s so far away from all the other data points, the correlation will be much lower.
In fact, when we include this data point the correlation drops from 0.93 to 0.58. The other limitation is that we can’t extrapolate, or make conclusions, about the data points outside the range of x-values that are in the sample.
For example, we know there is a correlation between the number of sugary beverages consumed and BMI for people that drink 2 to 12 sugary beverages per week.
But, we don’t know if there’s a correlation for people that drink 20 sugary beverages per week, because that’s outside the range of our sample data.
Finally, it’s important to keep in mind that if the assumptions for a Pearson’s correlation test are not met, we can’t be sure that the results of that test can be applied to the target population, so a Pearson’s correlation test shouldn’t be used.
Instead, we could use a non-parametric test called the Spearman’s correlation test, which doesn’t rely on parametric assumptions.
In short, the Spearman’s correlation test measures the strength of non-linear relationships, as long as the relationships are monotonic, meaning they are continuously positive or continuously negative, even if they are not in a straight line.
Alright, as a quick recap, the Pearson’s correlation test is a parametric test used to assess if there’s a correlation between two variables.
It’s important to keep in mind that the Pearson’s test works best for data that don’t have outliers, and that you can only make conclusions about the data points within the sample range.
The Pearson’s test assumes that the data are linear, independent and randomly sampled, normally distributed, and have equal variance.
And if these assumptions are not met, the Spearman’s correlation test can be used to determine if there’s a correlation two variables, as long as the relationship is monotonic.