Data Analysis Question: When dealing with significance levels, should I use p < .001 or p < .05? Also, should it change between tests or stay consistent throughout my analyses? (Matt, Chicago, IL)

In statistics, the level at which one seeks to find a significant p-value is known as "alpha". The most common levels of alpha are p < .05, p < .01, and p < .001. The decision about which to use is a difficult one, and is somewhat subjective. Essentially, alpha is the degree of chance that a researcher is willing to accept that the inferences they take from any given analysis are made in error. In other words, if I choose to use an alpha of .05, I'm accepting that there is an approximately 5% chance that I'll make assumptions from my results that may be an inaccurate representation of the population I am seeking to analyze. An alpha of .05 is the most commonly used alpha, although lower values of alpha (such as p < .001) are considered more conservative. There is no right or wrong answer about which to choose, although it is typically encouraged to keep the alpha statistic consistent across each analysis conducted in any given project, and to make the decision about which to use prior to running your analysis.