-By Tom Miller-
In the last few years there have been a number of high profile elections worldwide that pollsters got wrong, which has stained the face of the survey industry. The post-mortems on these elections identified a variety of problems unrelated to survey sampling or bad questions.
But even if the appearance of surveyor ineptitude is no more than a mirage, some people have gone sour on surveys. They fail to distinguish surveys from political polls. So what will the near future bring to surveys for local government?
To start, like the stock market, the survey’s reputation runs in cycles. Even now, there are no signs that recent incorrect political predictions have caused survey consumers to turn away. The private sector remains aggressively curious about what people think and do, so it can anticipate what they will buy. Government wants to get into the minds of constituents so it can predict how they will vote, what services they will use or if they will cooperate.
Here are ten specific improvements to expect from the survey research industry throughout 2018.
There was a lot of hand wringing about the election predictions that went awry. Survey experts all across the world considered and wrote about what went wrong. In 2018, pollsters are starting to pay much more attention to how surveys are done, who is responding and how the best math should be applied. More researcher caution, correct election calls and better press will elevate public confidence in surveys.
Visualization of findings will give the ability to view results in a number of different ways, rather than just PDF reports. It will also permit local governments and other stakeholders to treat results as “living” so they can parse results uniquely for different audiences and uses.
Panels and open-participation web surveys do not select potential respondents randomly like scientific surveys do. Instead survey participants arrive by their own convenience. However the speed and cost savings to survey online is so much greater than probability sampling that survey researchers will devise more ways to make panels and opt-in responses work.
Too many survey companies fail to let readers know how they select respondents, what the precise rate of response was, if and how they weighted the data and other aspects of the survey process. These processes are needed not only to evaluate the validity of the survey, but to be able to replicate the methods should others wish to survey the same population. The American Association of Public Opinion Research (AAPOR) has a Transparency Initiative that is creating healthy pressure on survey researchers to publicly describe their methods.
In fact, they already have. Industry-wide, telephone response rates fell from an average of about 36 percent in the early 1990’s to just nine percent in 2012. NRC observed address mail survey response decline from about 45 percent to about 28 percent within that same period.They have not declined since, and research demonstrates that even with single digit response rates, valid findings can be obtained.
Telling municipalities that residents feel safe in their community and that their sense of security is among the highest in the country has been essential for sustaining good cities. Linking that data to the comparative cost of the police force, along with known crime rates and numbers of sworn officers per 1000 citizens creates a third dimension of data. This helps managers decide where to put the next dollar and if their perceived strengths are backed up by local experience.
The big data movement to count traffic, transactions or tweets has a ton of useful applications. But they won’t replace knowing what’s happening between the city manager’s ears. Typically big data offer information about behaviors but not resident motivations, hesitations or intentions. Tracking the parking space turnover downtown won’t tell you if drivers admire, fear or loathe the downtown experience.
Survey results won’t just be interesting documents to shelve in the public library. We now have more interactive presentations of findings and better linkage of results to other relevant data. There is also keener understanding in management that residents expect their opinions to be put to good use. This all means that it will be more compelling for government managers to make the case for evidence-based policies, plans, budgets and segmented communications, making survey results essential to different stakeholder groups.
Many public sector survey clients are switching to annual surveys, rather than surveying once every two or three years. Internal processes are being created that make use of survey findings so staff have a system to appraise and act on what they hear from residents. Lower cost survey methods – opt-in web surveys and panels – will make it easier for local governments to afford more frequent monitoring of taxpayer opinion.
The most common surveys conducted by cities, towns and counties are the broad citizen surveys designed to monitor government performance and resident preference. More surveys will be conducted for deeper understandings of why residents rate services and conditions at the level given. A broader variety of surveys will also be done to understand the perspectives of employees, business owners and demographic segments of the community. Managers continue to appreciate that this 360 degree view of the community offers a better look at what is needed to thrive and encourages a more integrated approach to improvement.
A version of this article was originally published on PATimes.org.
Related Articles