Misc. Stats Topics Discussion > Validity of patterns observed in sample in entire population
Great question Ronald!
The degree to which the conclusions drawn from your sample are generalizable to the entire population are dependent on several things. Some of them would include:
- how was your sample selected? Was it random selection (most ideal for generalization to population)?
- Do the demographics of your sample generally reflect the demographics of your population or are certain segments (e.g. racial groups, age groups, or male/female) "over-represented"?
- the more your ratio of demographic groups reflect that of your population, the more generalizable your conclusions might be said to be
- How many trends/research questions are you examining?
- The needed sample size to obtain valid and reliable results in any analysis is related to the degree of precision that is required (chosen by the researcher) and by the number of variables that one is examining in any particular analysis. The more variables being examined in any analysis, the larger the sample size that is needed to take reliable conclusions from the results of that analysis.
Those are some key considerations, although there are definitely others. I hope that is helpful and feel free to post additional questions! Also, I'm curious to hear if others have thoughts on Ronald's question...
Jeremy Taylor
Thanks a lot, that was helpful.
The survey was mailed out to all households in the town; despite this, I am reluctant to generalize trends observed in the sample to the entire population. This is because several demographic groups have indeed responded to the survey more than others (higher incomes are one such demographic). Thus, I fear that the demographic breakdown of our group of survey takers does not reflect the demographic breakdown of the actual population.
We have been examining a handful of trends. Mostly, degrees of association were investigated (does answering a certain way to one question lead to an increased propensity to respond to a second question in a certain way?).
In particular, we were interested in learning if respondents that opposed the construction of a new road, were also more inclined to support measures that would preserve the town's open spaces (and those in the unbuilt lands outside the town but within the municipality). With a significance of >99%, I found that this was the case (I performed a Chi-square test). The phi-value for this association was .968, leading me to conclude that, in this case, there is such a strong association that it is likely that in the entire town's population, those who oppose the road's construction would indeed tend to support measures that would preserve open space.
Ronald Woudstra
Hi,
a town in Upstate NY has conducted a survey of around 1,800 of its inhabitants to gain insights into their opinion on certain land use issues.
The sample size is smaller than the entire town's population (more than 13,000), though fairly large given the comparatively small population in the entire town. For a graduate school advanced methods class, we have ran several analysis on the survey data that were obtained by surveying the approximately 1,800 inhabitants of the town. For our assignment, we are asked to elaborate upon the extent to which patterns observed in the sample are translatable to the entire town's population. In other words, are our conclusions -based on our SPSS analysis of the sampe- valid for the entire population?
It was my idea that if the data in the sample are distributed normally, the results observed in the sample will likely occur in the entire population as well. I believe that, if your results are distributed normally, you have a large enough sample size to conclude that whatever you observe in your sample is likely to occur in the entire population as well.
I would be interested in hearing other people's thoughts, and seeing whether people agree with my line of thinking, or whether perhaps there are other ways we can check if the results observed in our sample would likely occur in the entire population of the town.
Many thanks in advance!
-Ronald