Local governments need resident feedback that is representative and reliable. Here’s how cities and towns can avoid the most common mistakes in community survey research.
Your local government is ready to launch a community survey. Decision-makers, stakeholders, and residents are eager to see the results and how they’ll be used.
But turning resident survey results into meaningful action can be tricky. Even with the best intentions, pitfalls in the process can seriously diminish a survey’s effectiveness.
In this 30-minute webinar, you’ll see the most prevalent community survey mistakes and how to avoid them.
Polco Communications Director Angelica Wedell interviews Principal Research Strategist Michelle Kobayashi and Director of Account Management Alec Vice to help local governments understand the risks involved with community surveys and access resources for success.
They also explain how The National Community Survey® (The NCS®) works and guarantees the highest quality results.
A good community survey process ensures every resident has an equal chance to be invited to participate. Scientific surveys use probability sampling as the best way to achieve this.
Improving the community for everyone requires feedback from residents of all backgrounds. However, some resident groups are more likely to complete a survey than others. Traditionally underheard demographics include those whose first language is not English, racial and ethnic minorities, lower-income residents, younger adults, and renters.
Research shows that a large and growing number of residents prefer to engage with their local government digitally, especially since the COVID-19 pandemic. Online surveys are more convenient, accessible to many, cost-effective, and sustainable. That said, not all residents are online or feel comfortable with online surveys for personal privacy reasons.
Most people blame length for survey fatigue, and for good reason. But length is not the only factor that can burn out respondents—especially when it comes to community surveys. In fact, people are more willing to spend more time with surveys from their local government than from the private sector. They see community surveys as more important, interesting, and relevant to their quality of life.
These days, making community surveys too complex is a more common mistake. Lengthy questions with difficult wording or challenging options can quickly cause fatigue.
Multiple short surveys spaced out over weeks or months can also burden residents. The biggest challenge in survey research is to get residents to start a survey in the first place.
Survey questions are not all equal when it comes to burden. Open-ended questions (asking participants to answer in their own words) are cognitively challenging. The more open-ended questions, the lower the survey completion rate.
Inconsistent response scales are another major mistake. Mixing up scales confuses people, harming data quality.
It is all too common to see surveys asking residents to rate local government services they know little about. Surveying residents about services they don’t interact with—like internal government operations, risk management, or the quality of a jail system—leads to frustration and inaccurate data. Services used by only a small portion of residents should not be part of an omnibus community survey.
Another common mistake is using government jargon, acronyms, and other terms unfamiliar to most residents in the survey questions.
While the number of completed surveys is important, it’s not the only measure of quality. There are a number of other measures related to representation and methodology that matter significantly (including mitigation of coverage, non-response, and measurement biases).
Benchmarking is vital for comparing performance, but using outdated or irrelevant benchmarks skews results. If you are comparing your community’s results to the nation, make sure those data are recent and from a reliable source that uses large enough sample sizes and methods for accuracy.
Benchmarking against your own performance trends over time is also hugely beneficial. Using similar methods and recruitment strategies each time is the best way to guarantee comparability.
A city or town may feel reluctant to share survey data when ratings aren’t as high as hoped. But remember, there is no such thing as “bad” data (merely an opportunity to improve). It is vitally important for local governments to demonstrate transparency and publicize the results. Residents will respect your organization more for acknowledging their feedback and taking action.
Lack of vision is the number one reason why survey results may never see action. Knowing how your organization intends to use the results will help guide survey development, administration, and reporting. Also, communicating your plans to residents will increase survey response and representation and further build trust.
Local governments commonly use community survey data for strategic plans, budgets, policy development, and performance evaluation.
The easiest, most effective way to avoid the pitfalls is to work with a pro. They can offer guidance, data, and expertise that will help you track and use your findings. They can even conduct larger assessments (like The NCS) for you from start to finish. Polco can take care of all that legwork so you can focus on moving your community forward.
The National Community Survey® (The NCS®) stands as the most reliable and useful survey solution for local governments. It captures resident feedback across ten key facets of community livability, ranging from safety and economy to inclusivity and health (to name a few). Backed by 25+ years of research and the largest public opinion database of its kind, The NCS ensures scientifically valid, representative, and actionable insights.
Related Articles