Survey research is always a balance between the ideal survey design and the available resources. If it were feasible to survey every resident in a community, we would! While it is possible to do so within some small communities, in most jurisdictions, it would just be too expensive to send survey invitations and/or hard-copy questionnaires to every resident. Instead, we select only a portion—called a sample—of a community’s residents to participate in the survey. Sampling refers to the method by which survey recipients are chosen; the sample refers to all those who were given a chance to participate in the survey.
The first step in the sampling process is to determine from what source the sample will be selected. This source—called the sampling frame—must include as large a proportion as possible of the desired target population. The sampling frame may differ across different types of surveys, based on the specific target population and outreach method used for each. For The National Community Survey™, our target population is adults 18+ who live within the community’s boundaries, and we generally mail invitations to survey recipients. Thus, for this survey, our sampling frame is the most comprehensive list of addresses in a community, which comes from the United States Postal Service Delivery Sequence File (the list used by mail carriers to deliver mail).
After the sampling frame is chosen, we take a few additional steps to target the sample to the desired boundaries. First, we use a strategy known as systematic sampling (which closely mirrors the results of random sampling) to select every nth address in the zip codes that serve a community, usually selecting between 10,000 and 20,000 addresses. These selected addresses are then geocoded according to boundary/GIS data (either provided by you or based on other sources), to eliminate any addresses that fall outside the desired boundaries. From the remaining addresses, we randomly select a number of addresses to create the final sample. During this final random selection, we often oversample multi-family homes (defined in this case as addresses that contain a unit designation, such as #102 or Apt B), because in general, those in multi-family housing units often have other characteristics that are associated with lower response rates, such as being occupied by younger or lower-income residents.
When survey recipients are chosen in this way, the sample is referred to as a probability sample, because every community member has a known probability of being selected to participate in the survey. Probability sampling has been the gold standard for survey research for a long time, but as response rates continue to decline, Polco is actively exploring other approaches (including non-probability sampling methods) that will achieve responses that are representative of the entire community. Our outreach methods and sampling processes will continue to evolve, using survey research best practices to bring the representative voice of the community into public decision-making.