Why Paper Surveys Still Hold Value in the Digital Age

by Jessie O'Brien on April 28, 2023

man taking a paper survey

Pretty much everything is done online. Why do community surveys still require paper?

Paper surveys were once the gold standard in survey research. Things are changing, but just like the satisfying static of a vinyl record, old formats like paper still offer value that newer methods cannot.

“We’ve tried a couple of different approaches and we’ve learned some things,” Polco Director of Survey Research Jade Arocha said.

Let’s take a look at past methodologies to show you how we arrived at where we are today. Then let's take a look at where we plan on going in the future.

Death of the Dial Tone

The phone came along 5000 years after the invention of the written word. But the landline did not have the same lasting power as print. The survey industry shifted away from phone calls years ago for a number of reasons.

Community surveying requires that residents live in the area being researched. Area codes are no longer a reliable indicator of where a person lives since people can move states with their cell phones. Landlines are obsolete. And since most unknown phone numbers come from passionate strangers who desperately want you to extend your car warranty, people don’t answer their phones anyway.

The Problem With Paper Surveys

Unlike an area code, a home address does reveal where a person lives, which is why mail is suitable for surveying.

In the past, Polco’s National Research Center sent each potential respondent a postcard that notified the recipient that a survey packet was on its way. Then the packet would arrive, which was followed up by a second survey packet.

“That second wave increases response rate, but all of that paper, all of that postage, is expensive. It’s not environmentally sustainable,” Arocha said. “Then governments have to pay for the data entry team to enter all of those responses. It’s why paper-based surveying is becoming slightly outdated.”

So, why do we still use paper?

404: Online Issues

Polco started offering an online option years ago. All mail materials have a weblink for people who prefer a digital method. But we found that this method could not fully replace mail-in surveys.

We A/B tested the three-part mailing method (postcard, survey, then another survey) against an online-only option. For the online-only group, the randomly selected person would receive two postcards with weblinks to the assessment (they didn’t receive any paper survey packet in the mail).

Survey Illustration@2x Both groups were tested within the same community. “It’s not an apples to apples comparison unless you are trying both methods in the same jurisdiction,” Arocha said.

The first issue with online-only is some communities may have better access to the internet than others. Or some communities may have an older population or other demographic who are less willing to interact online.

“It was really hard to predict how many people would respond to each method,” Arocha said. “It was overcomplicating the approach.”

The web-only version created another problem. We list a phone number on the mailing materials for people who have questions. For the online-only group, local governments were inundated with tech support inquiries, which wasn’t practical.

The major benefit is the online-only cost is significantly lower—no paper, no postage, no paid data entry team. But the response rate is also lower. In some places it was much, much lower. This is an issue when government leaders want survey results that represent their entire community.

The Solution: Online and Paper Together

Now we recommend a hybrid approach: one postcard with the weblink, followed by one paper survey packet.

“It’s the best of both worlds,” Arocha said. “It allows people to respond online, it lowers costs, it is more environmentally sustainable, because it is one packet instead of two, and it enhances equity for people who don't have internet access.”

It’s important to note that the paper vs. online-only methodology question only applies when representation is the goal. There are plenty of online-only surveys that serve a valuable function. And plus, online often reaches more younger people, which has long been a challenge in the survey industry.

At Polco, we encourage governments to form an online subscriber base, so organizations have a built-in pool of residents willing to offer feedback quickly. Digital methods also completely eliminate the need for a data entry team since answers are automatically collected online.

But the survey industry isn't ready to move completely online just yet. Plus, using both provides more information to analyze. Polco’s goal is to compare both online and paper datasets to find out how disparate they are, and present clearer, more robust insights to local governments.

“All signs point to online surveying absolutely being the wave of the future,” Arocha said. “But we can confidently say we are not at the point where we can eliminate paper surveys. They certainly serve an important function and probably will continue to for a while.”

Learn more about Polco’s Representative Surveys

Polco benchmark comparison surveys allow you to compare your community with municipalities around the nation. Polco’s flagship products give government leaders a clear and representative view of their residents’ opinions. The results empower decision-makers to align their priorities with what matters most to community members.