Standard error of the mean (Central limit theorem)

Definitions & Key takeaways

Central limit theorem states that if the desired data is obtained repeatedly from random samples and the mean is calculated for each sample, these means will form a normal Gaussian curve.This curve will always be normal regardless to the shape of the original curve. The standard deviation of this curve is called the standard error of mean.The standard error of mean does not measure the dispersion of data but measures how much the sample represents the population. It is directly proportional to standard deviation and inversely proportional to sample size.
Watch video only
Let’s say you ask 1000 men for their weights and then plot those weights on a histogram, which is a type of plot that shows the distribution of measurements or data.
So let’s say that the majority of men weighed the same as the average - which in this case might be 170 pounds, or around 77 kilograms - while fewer men weighed a little bit higher or a little bit lower than the average, and even fewer men weighed much higher or much lower than the average.
If we draw a curve over the top of our histogram, we get the normal distribution curve, which is also called the bell curve, because it’s shaped like a bell.
The bell curve is symmetrical, with half the data on the left of the average and half the data on the right side of the average.
The area under the bell curve is equal to 1, or 100%, with the highest percentage of data in the middle section and the lowest percentage of data in the outer tails of the curve.
Typically, for population data, the average point in a bell curve is labeled with the greek letter mu, and mu refers to the mean, median, and mode, because when data are normally distributed, the mean, median, and mode are all equal to each other.
The standard deviation is a measure of how spread out the data are from the average, and for population data it’s represented by the lowercase greek letter sigma.
For example, let’s say the standard deviation of weight for our sample of men is 29 pounds, or 13 kilograms. In a normal distribution, 68 percent of the data are found within one standard deviation.
That means that 68 percent of men will weigh somewhere between 170 minus 29, or 141 pounds, and 170 plus 29, or 199 pounds.
Also, 95 percent of the data are found within two standard deviations - so, since 29 times 2 is 58, then 95 percent of men will weigh somewhere between 170 minus 58, or 112 pounds, and 170 plus 58, or 228 pounds.
Finally, 99.7 percent the data are found within three standard deviations, and since 29 times 3 is 87, 99.7% of men will weigh between 170 minus 87, or 83 pounds, and 170 plus 87, or 257 pounds.
This is called the empirical rule, or the 68-95-99.7 rule. Now, the shape of the bell curve depends on the size of the standard deviation.
A small standard deviation, like if it was only 5 pounds, tells you that most of the data are clustered around the average - and this makes the bell curve very tall and skinny.
On the other hand, a large standard deviation, like if it was 50 pounds, tells you that most of the data are way above and way below the average - and this makes the bell curve look very wide and flat.
It’s also possible that the population of 1000 men have a skewed distribution instead of a normal distribution, meaning one tail of the bell curve is longer than the other.
A right-skewed distribution means that the right tail is longer than the left tail, and a left-skewed distribution means that the left tail is longer than the right tail.
Typically, when the distribution is skewed, the mean, median, and mode are not equal. Oftentimes it’s impossible to collect measurements from every single person in the population, so we choose a sample which is basically a small number of people that we think represent the larger group.
As a general rule, if we collect the sample randomly, meaning people are chosen solely by chance, then we expect that sample to have similar characteristics - like the same distribution of weight - as the population they’re chosen from.
And if the two groups have similar characteristics, we also expect that the mean and the standard deviation to be the same in the two groups.
For example, let’s say we randomly take a sample of 50 men from the total population of 1000 men. If the population has a mean weight of 170 pounds and a standard deviation of 29 pounds, we also expect the sample to have a mean weight of 170 pounds and a standard deviation of 29 pounds.
But in some cases the sample we collect won’t have a mean of exactly 170. For example, a random sample of people might weigh more than the population mean, so the sample mean will be higher than the population mean.
Other times, most people in the sample will randomly weigh less than the population mean, so the sample mean will be lower than the population mean.
In fact, if we collect thousands of random samples of 50 men each, we’ll get a whole distribution of sample means - and this is called the sampling distribution.
The cool thing is, the sampling distribution always ends up as a normally distributed curve, with an overall sample mean that’s the same as the population mean, so 170 pounds!
This phenomenon is called the Central Limit Theorem, and it states that you’ll always get a normally distributed curve if you take multiple random samples from the population, and that the sample mean will be the same as the population mean.
To get a normally distributed curve though, the original population curve has to be normal; or, if the population curve is skewed, then the sample size - or the number of people in each sample - has to be large enough, which usually means more than 30 people.
One important thing to know is that the Central Limit Theorem only works if we use random sampling with replacement - meaning none of the men are ever completely removed from the population, so some of the men in the population might be picked twice, three times, or many times.
Now, whether the shape of the sampling distribution is tall and skinny or flat and wide out depends on the standard error of the mean - or simply, the standard error - which is a measure of how spread out the sample means are from the true population mean.
The standard error is calculated by dividing the standard deviation of the population by the square root of the sample size.
So, if we have a standard deviation of 29 and a sample size of 50, then the standard error is 29 divided by the square root of 50, which equals 4.1 pounds.
This means that, on average, a sample mean will be 4.1 pounds away from the population mean. If the standard error - 4.1 pounds - is small in comparison to the population mean - 170 pounds, then the sampling distribution curve will be tall and skinny, but if the standard error is large, like 20 for instance, then the sampling distribution curve will be flat and wide.
The standard error is a good indication that your study results can or can’t be applied to the population of interest. Let’s assume again that the standard error was 20.
This would mean that, on average, your study sample is likely to have a mean that’s around 20 pounds above or 20 pounds below the population mean.
And, if your sample mean is very different from the population mean, it’s likely that the sample’s other characteristics are different from the population as well - like the sample might eat healthier or exercise more than the population, which makes them weigh less.
When the sample and the population have very different characteristics, we say the study has low external validity. The standard deviation and the standard error are similar and are confused for each other, but they have distinct differences.
The standard deviation is used to describe how far an individual measurement is from the population mean. For example, a standard deviation of 29 pounds means that on average, a person in the population weighs 29 pounds more or less than the population mean of 170 pounds.
On the other hand, the standard error is used to describe how far a sample mean is from the population mean. For example, a standard error of 4.1 means that on average, a sample mean will be 4.1 pounds away from the population mean.
Finally, remember that the standard error will always be less than the standard deviation, since you divide the standard deviation by the square root of the sample size in order to get the standard error.
Alright, as a quick recap. The standard deviation tells you how far each individual measurement is from the population average, and it determines is the curve will be tall and skinny or wide and flat.
The Central Limit Theorem states that if you take multiple samples from the population and plot the mean of each of those samples, you’ll get a normally distributed curve - called the sampling distribution - that has the same mean as the population.
The standard error is calculated by dividing the standard deviation by the square root of the sample size, and a small standard error is a good indication that the sample is representative of the population.