Confusing Stats Terms Explained: Residual

When I hear the word "residual", the pulp left over after I drink my orange juice pops into my brain, or perhaps the film left on the car after a heavy rain. However, when my regression model spits out an estimate of my model's residual, I'm fairly confident it isn't referring to OJ or automobile gunk...right? Not so fast, that imagery is more similar to it's statistical meaning than you might initially think.

In statistics, a residual refers to the amount of variability in a dependent variable (DV) that is "left over" after accounting for the variability explained by the predictors in your analysis (often a regression). Right about now you are probably thinking: "this guy likes the word "variability" way too much, he should buy a thesaurus already!"

Let me try again: when you include predictors (independent variables) in a regression, you are making a guess (or prediction) that they are associated with the DV; a residual is a numeric value for how much you were wrong with that prediction. The lower the residual, the more accurate the the predictions in your regression are, indicating your IVs are related to (predictive of) the DV.

Related Content:

---

Confusing Stats Terms Explained: Heteroscedasticity

Confusing Stats Terms Explained: Multicollinearity

Confusing Stats Terms Explained: Standard Deviation

Keep in mind that each person in your sample will have their own residual score. This is because a regression model provided a "predicted value" for every individual, which is estimated from the values of the IVs of the regression. Each person's residual score is the difference between their predicted score (determined by the values of the IV's) and the actual observed score of your DV by that individual. That "left-over" value is a residual.

Like the imagery of the orange pulp, a statistical residual is simply what's left over from your regression model. They can be used for many things, such as estimating accuracy of your model and checking assumptions, but that is a chat for another time...

Editorial Note: Stats Make Me Cry is owned and operated by Jeremy J. Taylor. The site offers many free statistical resources (e.g. a blog, SPSS video tutorials, R video tutorials, and a discussion forum), as well as fee-based statistical consulting and dissertation consulting services to individuals from a variety of disciplines all over the world.

Comments

[DudeBro]

THANK YOU FOR BEING SO NICE AND SMART.
-Northwestern PhD student who feels really, really stupid in stats class

[Jeremy Taylor]

Glad I could be helpful, DudeBro!

[Mark]

Thanks for this series of plain-English explanations/definitions of statistical concepts. I was wondering though if you stopped at No. 8? Or are the 7 others somewhere else? I can't see them in this blog. Would want to know and read more of the items in your list.

[Jeremy Taylor]

Thanks for your post, Mark! To be honest, I got distracted away from the list, due to user feedback and requests to produce tutorial videos. However, I've received a few comments/questions about the list in the last few weeks, in particular, so I've been considering resuming that series!

I've also toyed with the idea of posting a interactive poll to elicit the users' help with choosing the remaining stats terms (and choosing what order they are in)... What does everyone think?

[Subramanyam]

Excellent. Wish text books were as clear as your explnantions. Please do write a text book

[Ariel]

Jeremy this is excellent and I hope you continue with the series. As a TA at my university I have directed students to your blog for further reading and they have come back to me ecstatic with how easy it is to understand your explanations.

[Jeremy Taylor]

Thank you, Ariel! Your encouragement means a lot to me, as it is exactly why I started the blog in the first place!

[Margaret]

Jeremy, I've just found your blog, and I'd like to add my voice to those hoping you'll continue the top 10 confusing stats. The explanations you've posted are great!

[Jeremy Taylor]

Thanks Margaret! Thanks for everyone's kind words. I intend to continue to try. To post and answer questions, but the rate with which I can reply has decreased, because I am currently on clinical internship/residency. However, I'll continue to respond as quickly as I can and I thank everyone for their patience!

[Tim]

Thanks for a super clear, super useful explanation.

[Jeremy Taylor]

I'm glad you found it useful, Tim!

[Kevin]

Seems like a lot of good information, but VERY hard to read on a mobile device. Pick a better font.

[Margaret]

Nice!!!

[Ayla]

Thank you so much Jeremy, there is absolutely no residual left from my inital total confusion about residuals :). I will keep reading what you wrote about other topics! Thanks again for your effort and compassion.

[Jeremy Taylor]

I'm really glad to hear it was helpful, Ayla!

[bryan]

thanks this helped!!!

[Jeremy Taylor]

Glad it was helpful!