Avoid These Top Ten Common Community Survey Mistakes

Local governments need resident feedback that is representative and reliable. Here’s how cities and towns can avoid the most common mistakes in community survey research.

Your local government is ready to launch a community survey. Decision-makers, stakeholders, and residents are eager to see the results and how they’ll be used.

But turning resident survey results into meaningful action can be tricky. Even with the best intentions, pitfalls in the process can seriously diminish a survey’s effectiveness. 

In this 30-minute webinar, you’ll see the most prevalent community survey mistakes and how to avoid them.

Polco Communications Director Angelica Wedell interviews Principal Research Strategist Michelle Kobayashi and Director of Account Management Alec Vice to help local governments understand the risks involved with community surveys and access resources for success. 

They also explain how The National Community Survey® (The NCS®) works and guarantees the highest quality results.

The Ten Most Common Community Survey Mistakes and How To Avoid Them

1. The survey outreach is not inclusive.

A good community survey process ensures every resident has an equal chance to be invited to participate. Scientific surveys use probability sampling as the best way to achieve this. 

• Why It Matters: Without inclusive outreach, survey results may not reflect the true composition of the community.
• How The NCS Works: The NCS uses both probability and opt-in methods to ensure the highest quality results and optimize cost. 
  
  

2. Responses don’t reflect the whole community.

Improving the community for everyone requires feedback from residents of all backgrounds. However, some resident groups are more likely to complete a survey than others. Traditionally underheard demographics include those whose first language is not English, racial and ethnic minorities, lower-income residents, younger adults,  and renters. 

• Why It Matters: To serve the entire community, feedback must come from everyone—not just the vocal minority.
• How The NCS Works: The NCS uses multi-mode outreach, offering surveys online, by mail, and in multiple languages.  It also weights responses to align with community demographics and ensure accurate and representative results.
  
  

3. The survey does not capitalize on technology.

Research shows that a large and growing number of residents prefer to engage with their local government digitally, especially since the COVID-19 pandemic. Online surveys are more convenient, accessible to many, cost-effective, and sustainable. That said, not all residents are online or feel comfortable with online surveys for personal privacy reasons.

• Why It Matters: More and more residents prefer to take surveys easily online. This trend will only increase with time. Technology also allows for convenient dashboards and reports. 
• How The NCS Works: The NCS uses a hybrid approach, offering both online and mail surveys to ensure accessibility for all residents. A part of the platform, Polco’s technology also incorporates results with curated data dashboards and AI tools.
  
  

4. The survey requires too much effort.

Most people blame length for survey fatigue, and for good reason. But length is not the only factor that can burn out respondents—especially when it comes to community surveys. In fact, people are more willing to spend more time with surveys from their local government than from the private sector. They see community surveys as more important, interesting, and relevant to their quality of life. 

These days, making community surveys too complex is a more common mistake. Lengthy questions with difficult wording or challenging options can quickly cause fatigue. 

Multiple short surveys spaced out over weeks or months can also burden residents. The biggest challenge in survey research is to get residents to start a survey in the first place. 

• Why It Matters: The harder a survey is to complete, the more response quality suffers, and the more likely people will abandon it.
• How The NCS Works: The NCS keeps surveys concise (8–12 minutes) and focuses on relevant topics like safety, economy, and health. Also, questions are simple, clear, and easy to answer. This is proven to minimize cognitive burden and help prevent drop-off.
  
  

5. Question types and formats are too complicated.

Survey questions are not all equal when it comes to burden. Open-ended questions (asking participants to answer in their own words) are cognitively challenging. The more open-ended questions, the lower the survey completion rate. 

Inconsistent response scales are another major mistake. Mixing up scales confuses people, harming data quality. 

• Why It Matters: Surveys must be easy to understand and complete to yield high-quality results.
• How The NCS Works: The NCS limits open-ended questions and keeps scales similar and consistent. The range of response options is fewer than five, and choices are labeled with descriptive words (e.g., “Excellent” or “Good”) instead of just numbers. It also includes a “Don’t Know” option where appropriate.
  
  

6. Questions go beyond the average resident’s scope of knowledge.

It is all too common to see surveys asking residents to rate local government services they know little about. Surveying residents about services they don’t interact with—like internal government operations, risk management, or the quality of a jail system—leads to frustration and inaccurate data. Services used by only a small portion of residents should not be part of an omnibus community survey. 

Another common mistake is using government jargon, acronyms, and other terms unfamiliar to most residents in the survey questions. 

• Why It Matters: Respondents can only provide meaningful feedback on topics they would have an opinion about.
• How The NCS Works: The NCS avoids jargon and focuses on areas that impact daily life, such as safety, mobility, and the economy, ensuring questions are relevant and actionable. 
  
  

7. Survey quality is judged solely by response numbers. 

​​While the number of completed surveys is important, it’s not the only measure of quality. There are a number of other measures related to representation and methodology that matter significantly (including mitigation of coverage, non-response, and measurement biases).

• Why It Matters: The number of responses alone does not guarantee a representative result.
• How The NCS Works: The NCS prioritizes representativeness over sheer numbers, using scientific sampling and demographic adjustments. Questions are also written and formatted in ways that produce the highest-quality data.
  
  

8. Using unreliable benchmarks.

Benchmarking is vital for comparing performance, but using outdated or irrelevant benchmarks skews results. If you are comparing your community’s results to the nation, make sure those data are recent and from a reliable source that uses large enough sample sizes and methods for accuracy.  

Benchmarking against your own performance trends over time is also hugely beneficial. Using similar methods and recruitment strategies each time is the best way to guarantee comparability.  

• Why It Matters: Valid comparisons create meaningful context and set realistic expectations. 
• How The NCS Works: The NCS represents the opinions of well over 30 million Americans nationwide. Its database is the largest of its kind, reporting resident perspectives on community livability. The NCS offers both national and customized benchmarks tailored to your community. Most local governments repeat The NCS every year or two, as the survey reports performance trends over time. Also, by leveraging reliable probability samples, The NCS ensures similar types of respondents over time.
  
  

9. Survey results are available for internal eyes only.

A city or town may feel reluctant to share survey data when ratings aren’t as high as hoped. But remember, there is no such thing as “bad” data (merely an opportunity to improve). It is vitally important for local governments to demonstrate transparency and publicize the results. Residents will respect your organization more for acknowledging their feedback and taking action. 

• Why It Matters: Sharing results shows residents their voices matter and is one of the greatest ways to build a stronger sense of civic trust.
• How The NCS Works: The NCS offers clear, visual reports that are easy to share. Polco researchers can also present results virtually or in person at city council meetings. 
  
  

10. Failure to take action with the results.

Lack of vision is the number one reason why survey results may never see action. Knowing how your organization intends to use the results will help guide survey development, administration, and reporting. Also, communicating your plans to residents will increase survey response and representation and further build trust. 

Local governments commonly use community survey data for strategic plans, budgets, policy development, and performance evaluation. 

• Why It Matters: Residents want to see their feedback lead to meaningful change.
• How The NCS Works: Polco offers Next Steps Workshops to help governments translate their survey data into action.
  
  

Do you recognize any of these ten community survey mistakes?

The easiest, most effective way to avoid the pitfalls is to work with a pro. They can offer guidance, data, and expertise that will help you track and use your findings. They can even conduct larger assessments (like The NCS) for you from start to finish. Polco can take care of all that legwork so you can focus on moving your community forward.

Why The National Community Survey Is the Gold Standard

The National Community Survey® (The NCS®) stands as the most reliable and useful survey solution for local governments. It captures resident feedback across ten key facets of community livability, ranging from safety and economy to inclusivity and health (to name a few). Backed by 25+ years of research and the largest public opinion database of its kind, The NCS ensures scientifically valid, representative, and actionable insights. 

Related Articles

• How Elk Grove Improves Services With a Data-Driven Mindset
• Celebrating Excellence in Local Government: The 2024 Voice of the People Awards Winners and Finalists
• Community Livability Measures Governments Should Track (and Make It Easy)