Trade Your Stats "Truths" for Stats Arguments...

Warning, this blog will be short, sweet, and a bit pithy. The two most common questions that I receive about statistical analyses, no matter what kind or purpose, is: "Am I doing it right?" or "Am I allowed to...(fill-in a variation of a common analysis here)?" My response to these questions is usually: "Sure, you can do whatever you want, but what will it mean if you do?" I've said for a few years now that I don't see statistics as being about find truths, but instead I see it as being about building arguments. The critical things is that you understand the impact of your statistical decisions.

Just like your high school math teachers told you, you must learn the rules before you can know when it is useful to break them. The same is true when building arguments with statistics. There aren't many things you can do when running a statistical analysis that are inherently "wrong". However, it is critical that you remain aware of the consequences or your statistical decisions. Next time you find yourself wondering if you are "allowed to" or are "supposed to" do something in an analysis that is not what is typically done, stop yourself from asking those questions and consider what would be different if you did it, and what would it mean for your interpretation?