Curious about the difference between probability-based versus open participation surveys? Here are some details:
What's an open participation survey and what are the benefits?
An open participation survey is offered to all residents in a community. This method is important because it provides engagement opportunities to all, thus often providing a higher number of completed responses overall and for population sub-groups. It can be less resource intensive and require less time from administration to actionable data. It is programmed as an online survey with invitations sent widely through as many communication channels as possible (social media, email lists, newsletters, media reporting, etc.). Open participation surveys also can benefit from snowball sampling whereby existing survey respondents can recruit their friends and acquaintances and response grows like a rolling snowball. This sampling method can be used to elevate general awareness of the research effort and increase overall response and the response from those who do not traditionally participate in surveys.
What's a probability-based survey and what are the benefits?
A probability-based survey uses a sampling method where every eligible person or household in a community has a known chance of being selected to participate. Respondents are selected at random from a comprehensive list (usually of residential addresses) as an effort to reduce bias. Probability-based sampling is considered the gold standard for survey methods, although with declining response rates and the advent of new outreach and weighting methods, this status may change.
Can I do a probability-based survey and an open participation survey at the same time?
Yes, in fact it is always advisable to include an open participation survey when implementing a survey with a probability-based sample. The probability-based method ensures that every resident has an equal chance of being selected, but most are not selected. The open participation works to increase the number of completed surveys and also ensures that the whole community feels included and everyone has a chance to be heard.
How do I know which type of survey to choose?
A number of factors play into the choice between a probability-based survey and an open participation one. Timeline, resources, and ability to do adequate outreach all may affect the decision. A probability-based survey will require more resources (generally money) to mail invitations to the randomly selected households, as well as a longer timeline for survey completion; however, these results will likely be more representative of the opinion of the entire community. The open participation survey can be completed more quickly, but more burden may be placed on local government staff to do the outreach, and the results may not be as representative of the community. Individuals that are already more engaged with your local government are more likely to participate, and thus may be more like those from whom you already hear. However, with more effort invested in outreach, you may be able to reach the parts of your community that tend to be less likely to engage with local government and who may not respond to a typical probability-based survey. This additional effort spent initially reaching the community can help build your Polco subscribers and enable you to dig deeper on specific topics in the future.
Those looking to do performance measurement or one of Polco’s benchmark surveys often choose probability-based methods, along with an open participation survey, to include the entire community and begin building a panel of subscribers. Those who need quick input, crowdsourced ideas, or in-depth feedback exploring the reasons behind respondents’ opinions may be best served by open participation.
What do I do with the two types of data?
There are options! The data can be treated separately, using the probability-based survey to gain insights from the representative sample and the open participation to better understand the most vocal opinions in the community. At Polco, we also have the statistical tools to blend the samples together to create a larger, stronger set of responses. We can advise which approach we recommend based on specific results, so feel free to ask!
