Reshaping the Research Landscape: Public Opinion and Data Science

by NRC on July 7, 2016

The annual American Association for Public Opinion Research (AAPOR) conference is a great avenue to check in on the latest industry research, confirm best practices and see where innovations are being made. After attending the 71^st Annual AAPOR Conference, National Research Center, Inc. (NRC) Vice President Michelle Kobayashi and Director of Research Sonya Wytinck share their biggest takeaways.

Web Surveys

web-computer_pixabay_CC0

Web surveying still has a way to go before it can create representative samples of the population. Although a multitude of federal agencies have begun to experiment with web surveys and on-line panels as a way to increase response rates and decrease survey costs, there are some inherent differences between those who participate in the web panels and the general population. Research to understand, and correct for, those differences is in its first stages but there are a lot of innovative ideas being pursued. At this point, the most important thing we need to do is to understand the confounding variables in our web data, those things that are related to our topic of interest where people in a web sample may differ in opinion from the general population.

On average, web panelists are likely more tech-savvy and are early adopters; does that make them more ready for your new policy? More willing to trust? Web surveys can be great to get a quick read on things and can work for highlighting differences between groups (e.g., are men more skeptical than women about your program?). In the end it is important to know when the quick read will do the job (at a lower cost) and when you should collect a more representative sample of the general population.

Telephone Surveys

phone call_unsplash_CC0

Telephone survey samples should include mostly cell phone numbers, as most people have cell phones and shrinking numbers have landlines. This has some practical implications related to laws about calling cell numbers, which can increase survey costs. Overall, the number of people who do not answer survey calls is increasing for both cell and landlines, while non-working phone numbers are increasing for landlines and decreasing for cell phones. This means higher call volumes are needed to reach willing survey participants: a lot more calls for landlines and a little more for cell phones. There is, however, some evidence that lower response rates have not adversely affected representativeness (once the data is weighted).

Mail Surveys

writing-paper_unsplash_CC0

Mail surveys still have the best response rates, although overall response rates (from any method) are dropping. The key to maintaining response rates is to keep the survey as simple as possible. Survey titles must be broad, to make sure people don’t think “that doesn’t apply to me.” Question wording should be easy to understand and the questions should be as short as is reasonable (know that people will skim long questions). We recommend starting with easier questions – those that the most people will have a ready opinion about – to ease respondents into the survey.

Big Data

looking at data_pixabay_CC0

Big Data is all the rage in the private sector and this is spilling over into government. Learning how to best capture, clean and report these data is the real challenge. With the advent of new technologies and social media, the ability to harness and use secondary data will be a dominant theme in the coming years and decades. The pairing of secondary data with survey data holds much promise for the future.

Related Articles

National Research Center, Inc. (NRC) is a leading full-service survey research and evaluation firm focusing on the information requirements of the public sector, including local governments, health care providers, foundations and non-profit organizations. Visit our home on the Web at https://www.n-r-c.com//. Check out our media page for more news, tips and human-interest stories from NRC.