If you have dabbled in survey research and methodology, you’ve likely seen your fair share of industry jargon. Terms like “confidence interval” or “sample” tend to sprinkle scientific conversations and reports. Local governments that survey their residents want to understand how well the data reflects their entire community. So “Margin of Error” is a particularly important term for local leaders to consider, and one researchers and analysts are often asked about. NRC Survey Associate Jade Arocha has managed dozens of community surveys in cities and towns across the U.S. In this video, she explains what “Margin of Error” means and why it matters.
Let’s talk about why we need a “margin of error,” or a way to measure how well survey results represent a population. If we were to survey every person in a community, there would be no need for margin of error. But surveying every single resident in a city of thousands is very costly in terms of money, time and staff. Most local governments lack the resources to conduct a complete census of their residents. (And not everyone will respond to the questionnaire.)
So for most cities, we only survey a randomly sampled portion of the population. This gives us an overall sense of how residents think without numbering individuals. (You only need a spoonful to understand how an entire pot of soup tastes.)
How Margin of Error Works
For example, say we survey every person in a community, and find that 80 percent of them rate their quality of life as excellent or good. We know for sure that number reflects exactly 80 percent of the population because we have the data from every person. However, surveying a proportion of the population creates a range in accuracy.
So in this example, NRC’s methodology allows us to identify that 80 percent of the population rated their quality of life as excellent or good within a margin of error of five percent. That means the results sit very near 80 percent, give or take up to five percent. So somewhere between 75 and 85 percent of residents rated their quality of life as excellent or good.
Margin of Error Depends onthe Number of Responses
The margin of error is determined by how many responses we get. Say the population contains 5,000 adults. Receiving 300 responses will have a higher margin of error than 500, and 500 will have a higher margin of error than 1,000, etc. The lower the margin of error the better.
A higher response rate means a lower margin of error. So our recommended sample size is based on where we want that margin of error to be. NRC survey researchers take measures to produce data that are highly accurate, so you get results you can rely on.