Methods of regression analysis
Definitions & Key takeaways
There are a variety of methods of regression analysis, each with its own strengths and weaknesses. The most commonly used methods are linear regression, logistic regression, and Poisson regression.
Linear regression is used when the data is assumed to be linear in nature. Logistic regression is used when the data is assumed to be binary (e.g., success/failure, yes/no), while Poisson regression is used when the data follows a Poisson distribution, and is used for modeling count data.
There are four basic types of statistical analyses commonly used in epidemiological research, and the analysis you pick depends on two main criteria.
The first criterion is the type of data you have, which can be either individual data or binned data, which is also called group data.
So, for example, let’s say we want to know how many people out of 100 people developed lung cancer the past 5 years. With individual data, we have information about each person, so we can tell whether or not each of the 100 people developed lung cancer.
So let’s say that 6 people developed lung cancer. If we have individual data, we can look at the individual characteristics for each of those 6 people, like their sex, age, race, or past history of migraines, and we can compare them to the people that didn’t developed lung cancer.
On the other hand, if we have group data, we don’t actually know which specific individuals out of the 100 people developed lung cancer.
So even though we know that 6 people had them, we don’t know which 6 people they were or any of their individual characteristics.
The second criterion is the type of outcome or y-variable you’re measuring, which can be either quantitative, categorical, or time to event.
Quantitative variables have a numeric value, like a person’s forced expiratory volume, which is the total amount of air, in liters, that a person can exhale in a single forced breath.
A very fit person might have an FEV of 5, while a less fit person might have an FEV of 3. On the other hand, categorical variables have distinct levels.
For example, we could use a categorical variable to characterize if a person was diagnosed with lung cancer in the past five years or if they were not.
And finally, time to event variables describe how long a person was followed before the event or outcome occurred. For example, if we started following a person at age 50 and they developed lung cancer at age 53, then their time to event would be 3 years.
Now, one of the simplest and most widely used types of analysis is linear regression. Linear regression uses individual data, and the outcome variable is always quantitative, while the exposure variable can be either categorical or quantitative.
For example, let’s say we want to figure out if there’s an association between the number of cigarettes smoked and FEV, so we ask 100 people how many cigarettes they smoke in a day and then measure each person’s FEV.
In this study, the exposure is the number of cigarettes, so it’s quantitative, and the outcome is FEV, which is also quantitative.
Typically, we use statistical software to calculate the linear equation, and the software will provide b0 and b1, which are two numbers we can then plug into the equation y-hat = b0 + b1x1.
Y-hat is the estimated value for the outcome variable, which in this case is FEV, and x1 is the value of the exposure variable, so in this case that’s the number of cigarettes a person smokes.
So let’s say the software gives us a b0 of 4 and a b1 of negative 0.1, so the equation is y-hat equals 4 minus 0.1 times x1.
Now, b1 is the most important number for interpretation because it tells us the effect size, or how much the outcome variable changes for every one-unit increase in the exposure variable.
For example, a b1 of negative 0.1 means that, on average, the FEV will decrease by 0.1 liters per second for every one additional cigarette smoked per day.
One important thing to know is that linear regression can be used in any type of study design as long as the two criteria of individual data and quantitative outcome variable are met.
The next type of statistical analysis is logistic regression. Logistic regression uses individual data, and the outcome variable is always categorical while the exposure variables can be either categorical or quantitative.
For example, let’s say we want to figure out if smoking more cigarettes increases the chance of lung cancer between the ages of 55-64.
So, we follow a hundred 55-year-olds that smoke and a hundred 55-year-olds that don’t smoke for 10 years, and compare how many of them develop lung cancer.
In this example, the exposure variable is whether or not a person smokes cigarettes, so it’s categorical; and the outcome variable is whether or not the person develops lung cancer, so it’s also categorical.
And more specifically, because there are only two levels for each variable, they’re called binary categorical variables.
Now, like linear regression, the statistical software will give us b0 and b1, and we can plug them into the same equation of y-hat = b0 + b1x1, but the interpretation of the beta-coefficients are different.
In logistic regression, the beta-coefficients represent the log-odds of the outcome occurring. For example, let’s say the software gives us a b0 of 0.05 and a b1 of 1.9, so the equation for the line would be y-hat equals 0.05 plus 1.9 times x1.
So, a b1 of 1.9 means that, on average, the log-odds of developing lung cancer for smokers is 1.9 times the log-odds of developing lung cancer for non-smokers.
Since the log-odds can be a confusing interpretation, we can also convert these numbers to regular odds by exponentiating them by a base of e.
For example, e to the 1.9 equals 6.7, so the odds of developing lung cancer for smokers is 6.7 times the odds of developing lung cancer for non-smokers.
Logistic regression can be used for any type of study, but the interpretation changes slightly depending on the study design.
Our example was a longitudinal cohort study, because we had a group of exposed individuals—those are the ones that smoked—and a group of unexposed individuals—those are the ones that didn’t smoke—and followed them over time.
This type of study design allows you to measure the incidence or the risk, which is the number of new cases that occur over a certain period of time.
Using logistic regression, we then calculate what’s called the risk odds ratio. On the other hand, logistic regression can also be used in case-control studies, which is where you compare the history of two groups of people—those that have a certain outcome, called cases, and those that don’t have a certain outcome, called controls—to see if they’ve been exposed to different things.
So, for example, we could’ve looked at 100 people that had lung cancer, which would be the cases, and 100 people that don’t have lung cancer, which would be the controls, and then compare how many people in each group smoked cigarettes in the past ten years.
Now, in case-control studies, we can’t measure the incidence, since we’re selecting people that already have the outcome.
Instead, we’re measuring the prevalence, or the number of people that already smoked cigarettes before we started measuring them.
In case-control studies, we can use logistic regression to then calculate the prevalence odds ratio. Another type of logistic regression is called conditional logistic regression, and this is specifically used in matched case-control studies.
Matching is when the cases and controls are both selected based on a certain characteristic, like age, sex, race, socioeconomic status, or occupation; and it’s often used to ensure that the case and control groups are similar.
For example, you could match by gender to ensure that 50 percent of cases and 50 percent of controls are women. Conditional logistic regression produce matched odds ratios, which are calculated differently using statistical software but are interpreted very similarly to other odds ratios.
For example, a b1 of 1.5 would be interpreted as the log-odds of developing lung cancer for smokers is 1.5 times the log-odds of developing lung cancer for matched non-smokers.
Okay, the next type of statistical method is the cox regression, which is also called cox proportional hazards regression.
Cox regression uses individual data, and the exposure variable is categorical, while the outcome variable is time to event.
For example, let’s say we find 500 people who smoke and 500 people who don’t smoke, and follow up with them every year for the next twenty years.
At each yearly follow-up visit, we record whether or not the person has developed lung cancer or not, and if they do, we write down how many years we followed them before they developed the outcome.
To simplify, let’s just take five people as an example, and let’s say they’re in the smoking group. So, let’s say person 1 developed lung cancer after 5 years, person 2 developed it after 7 years, person 3 developed it after 10 years, person 4 developed it after 11 years, and person 5 never developed lung cancer.
So, we would say that the time to event for persons 1 through 4 is 5, 7, 10, and 11 years, while the time to event for person 5 is 20 years, since they were followed all the way through the entire study without developing lung cancer.
Oftentimes, these data are presented on a Kaplan-Meier plot, which is a type of stepwise plot with time on the x-axis and percent “survived”, or disease-free, on the y-axis.
In this case, each step represents one person being diagnosed, and a flat line means that no one was diagnosed during that time.
Now, normally, both exposed and non-exposed groups are shown on the same plot, so let’s add the non-smoking group, and we’ll say that person 1 developed lung cancer at 8 years and person 2 developed it at 18 years, while persons 3, 4, and 5 did not develop it at all.
One thing to notice is that the line for the non-smoking group is above the line for the smoking group, which indicates that there was a higher percentage of people who were disease-free in the non-smoking group.
Alright, back to cox regression. So, the exposure variable is smoking or not smoking, which is a binary categorical variable, and the outcome variable is the time to developing lung cancer.
And hazard is defined as the instantaneous risk of an event occurring in a certain period of time. So, let’s say b1 equals 0.9, which means that the log-hazard of developing lung cancer for the smoking group is 0.9 times the log-hazard for the non-smoking group.
Just like in logistic regression, if we exponentiate b1, we get the hazard ratio. So, e to the 0.9 equals 2.5, which means that the hazard of developing lung cancer for the smoking group is 2.5 times the hazard for the non-smoking group.
Typically, cox regression is used to analyze data from longitudinal cohort studies, since you have to follow people over time in order to collect time to event outcomes.
Alright, now let’s switch gears and talk about binned or grouped data. One type of statistical analysis you could do with binned data is called Poisson regression, and—like cox regression—it uses time to event outcomes to analyze data from longitudinal cohort studies.
But the catch here is that, unlike cox regression, you don’t know the exact times to event, because you only have information about groups and not about individuals within each group.
This sometimes happens when you didn’t collect the data yourself and you can’t access the original data because of logistical reasons—like ethical, physical, or economic restrictions.
So, let’s say you want to find out the association between smoking cigarettes and time to lung cancer diagnosis in 200 people—100 people who smoked and 100 people who didn’t smoke—and everyone was followed for a maximum of 20 years.
Now, because it’s binned data, you have the total number of people who developed lung cancer in each group and the total amount of time that people were followed in each group, but you don’t know exactly how long each of the 200 people was followed.
If no one got cancer (the event), it would mean that every single person was followed up for the entire 20 years, and that total time would be 100 people times 20 years, or 2 thousand person-years, in each group.
But that’s pretty unrealistic, so let’s say that 14 people in the smoking group developed lung cancer and 2 people in the non-smoking group developed lung cancer.
And let’s also say that there’s a total of 1,200 person-years in the smoking group and 1,500 person-years in the non-smoking group.
From these numbers, we can figure out the incidence rate in each group, or the number of events divided by the person-time.
In the smoking group, the incidence rate is 14 divided by 1,200, or 0.012—which can also be stated as 12 cases of lung cancer per one thousand person-years.
In the non-smoking group, the incidence rate is 2 divided by 1,500, which equals 0.0013—and this can also be stated as 1.3 cases of lung cancer per one thousand person-years.
If we divide the incidence rate of the smoking group by the incidence rate of the non-smoking group, we get 0.012 divided by 0.0013, which equals 9.23.
And this is the incidence rate ratio, which means that the incidence rate of lung cancer in the smoking group is 9.23 times the incidence rate of lung cancer in the non-smoking group.
So how does this fit into Poisson regression? Well, statistical software uses Poisson regression to calculate the log-incidence rate ratio for you, which it gives as b1.
And this can be exponentiated to give you the incidence rate ratio. So even though it’s possible to calculate simpler incidence rate ratios by hand, it gets a lot more complex when you start adding in other variables, like age group or socioeconomic status, and that’s when Poisson regression comes in handy.
Alright, as a quick recap, there are four basic types of statistical analyses commonly used in epidemiological research, and the analysis you choose to use depends on two main criteria— the type of data you have, which can be either individual data and binned or grouped data, and the outcome or y-variable you’re measuring, which can be either quantitative, categorical, or time to event.
Linear, logistic, and cox regression all use individual level data, while Poisson regression uses binned data. In linear regression, the outcome variable is always quantitative, and the b1 can be interpreted as the change in the outcome variable for every 1 unit change in the exposure variable.
In logistic regression, the outcome variable is always categorical, and when the exposure is also categorical, the b1 can be interpreted as the log-odds ratio of the outcome for the exposed versus unexposed group.
And if you exponentiate it, e to the b1 is interpreted simply as the odds ratio of the outcome for the exposed versus unexposed group.
In cox regression, the outcome variable is time to event, and e to the b1 can be interpreted as the hazard ratio of the outcome for the exposed group versus the unexposed group.
Finally, in Poisson regression, the outcome variable is also time to event, and e to the b1 can be interpreted as the incidence rate ratio of the outcome for the exposed versus the non-exposed group.
No notes for this video yet
Try adding a note below