Beginning in 2024, Polco adjusted the standard (default) methodology for The National Community Survey; rather than mailing hard copy surveys to randomly selected households, we now mail postcards to randomly selected households inviting them to take the survey online.
We did not make this change lightly! We conducted our own research to first understand potential ramifications, and what we found supported this move toward digital survey data collection. Read more details about the study that led to this decision here.
As broadband access and internet usage continues to expand, conducting surveys exclusively online grows even more attractive, for many reasons. Compared to traditional mailed surveys, digital data collection:
...all while having no discernible effect on respondents’ demographics or responses to survey questions, according to our research!
From 2020 to 2022, Polco conducted nearly 100 implementations of The National Community Survey (The NCS) with a split sample methodology. In each of these communities, households were randomly selected to participate in the survey following our standard sampling process. A portion of these households were contacted using traditional mailed outreach (which included a hard copy paper survey), while the remainder were contacted using postcards with a link to the online survey (no hard copy).
Our in-house data scientists then analyzed these results and found that while response rates were found to be lower for the postcard-only invitations, the demographic profile of respondents AND survey responses were both nearly identical between the two groups.
These findings were presented in length at the 2022 national convention of the American Association of Public Opinion Research, and a more detailed written summary of the research and methodology can be found here.
During this study, Polco heard a few concerns from customers around two main topics: lower response rates and residents with limited internet access or familiarity.
While you can anticipate seeing a lower response rate with digital data collection than with traditional mailed survey outreach, we counteract that by increasing the sample size. Mailing postcard invitations to more households will help us continue to achieve a 4-6% margin of error at the 95% confidence level, maintaining the same precision and confidence in your results as in hard-copy survey implementations.
To address the concerns around internet access and usage, we strongly recommend including a phone number and email address on the survey invitations specifically for technical questions/assistance. Additionally, you could consider making paper surveys available for residents upon request, with staff individually mailing and manually entering the paper responses online afterward. We’d be happy to talk through these considerations, and any questions you may have about methodology, as you plan your next survey project.
Overall, our research gives us the confidence to recommend digital survey data collection, despite these minor drawbacks. As the world continues to grow more technologically advanced, we must update our methodology accordingly to meet people where they are. By adopting this digital surveying approach (in which residents receive a physical, mailed postcard invitation but complete the survey online), we can combat decreasing response rates and continue to provide you with trusted, unbiased data about your community.
I still would prefer to mail a traditional survey packet for my community survey—is that still available?
While the postcard-only outreach will be our default methodology, Polco will still offer a traditional survey packet mailing (containing a hard copy survey) for those who prefer! However, we will pass on the additional costs for this outreach methodology.