Getting Past “Huh?” The Not-So-Straight Line from Data to Decisions

-By Tom Miller-

A guy from Denver walks into a bar in San Francisco and is about to order his drink of choice – a Guinness Black – when he feels the ground start to tremble. At first he thinks it’s the large garbage truck he saw moving through the alley collecting recycled glass. The shaking seems to get stronger and the bottles behind the bar are quivering. He looks around. No one seems phased, but he knows he’s in an earthquake zone. He’s thinking, I could dive under a table and prove without a doubt that I am from out of town; I could step quickly out of the bar and have the bartender run after me for not paying the bill; I could yell, “Run for your lives, it’s an earthquake!” and be arrested for causing a disturbance.  Or be recognized as a local hero. “Now what do I do?”

What a great question. And it shouldn’t be reserved for rare geological events.  “Now what do I do?” is a question that researchers and consultants wish every decision maker would ask. You can’t be in the business of teaching or practicing research methods without eventually confronting the anemic use of research results by the decision makers for whom the results are presented.

Across decades, many pros in our business have wrung their collective hands over this problem – see Weiss, House, Patton and others about using research results.  Decision makers don’t often and never often enough rely on research to make policy decisions. Yes, when things are good, we can point to the reliance on research among inputs of politics, payments and predilections, knowing that the dispassion of research rarely trumps the passion of the other sources. What does it take to get decision makers to ask themselves about their research results, “Now what do I do?”

My colleagues and I provide survey results to city and county managers that start with what their residents think about the quality of community life, service delivery, public trust and civic involvement. Once they’ve received and had time to think about the data, I want local government managers to ask me, “Now what do I do?”

Getting community or national leaders to ask this question means that those leaders have gotten past foot (that punts), knee (that jerks) and gut (that reacts) conclusions to really engage the brain. Leaders that find themselves asking “now what do I do?” are not necessarily in a crisis but they are engaging what Daniel Kahneman calls System 2 thinking.  System 2 is the judicious parent of your family of cognitive functions, whereas System 1 is the flighty in-law quick to believe gossip and act on instinct.  If you think about it, with discerning System 2 processing, most of our daily decisions are made by habit.

We can lament our own common practice of relying on System 1 thinking most of the time, but we have to acknowledge that “1” is no lonely number. The entire human race (or at least those who have been subjected to experiments perpetrated by social psychologists) is predisposed to think this way.

To know what to do, it’s best to know what strikes you as a problem and then to determine what caused the problem (because if the problem is a garbage truck and not an earthquake, solutions vary). But putative causes often are ephemeral or illusory. It’s tough to figure out causes. Think about what caused your last cold? Still, if you even just speculate about a cause, say, for a poor opinion among residents of your community’s reputation, you are on the road to System 2 thinking and a reasonable solution. But don’t ignore the System 1 thinker in you. A manager that is disciplined enough to have a system for decision-making won’t be shooting blanks from the hip.

Here are seven actions that a decision maker who gets survey data can take once he asks (or asks you), “Now what do I do?”

We use these with managers who have commissioned surveys of their residents. These steps may be useful in many other arenas.

1. Listen: Think of the survey as an eavesdropping device. You get to hear what your residents are thinking about the most fundamental parts of the business you are in.
2. Feel: Once the survey results are in, take a moment to check how they sit with you. Rather than pretend that manager decisions are made strictly by the numbers and that emotion should play no part in the power of data, perhaps the first question a manager should ask of his survey data is not “does it make sense?” but “does it feel right?”
3. Think: Once you’ve identified what survey results resonate with you, examine them. Do they square with other data you have? Do they confirm what you and others have observed?
4. Convene: Managers are not in this alone. There are some, maybe many, staff who have more direct experience with every topic in your survey. Gather them together with a set agenda and reasonable time frame.  Identify three to five key areas for your jurisdiction to do something about – communicate, change services, analyze more deeply, convene additional partners. Define the outcomes sought, assign staff, create a timeline and identify metrics of success.
5. Empower: Send staff back to the departments they lead and have all or select line staff go through steps 1-4, but focused on those things that the department can do to support success in the 3-5 key areas identified by the executive team.
6. Experiment: Acting on data always involves some risk – as does failing to act. No one can be sure if a best practice in a text book will result in the intended outcome in your jurisdiction. One way to reduce risk is to test the ideas that come from staff.
7. Evaluate: Actions in place, you have to revisit what you’ve done to determine if you are getting what you wanted. Whether you’ve gone the way of a formal experiment or simply have instituted a new jurisdiction-wide policy or program, assess success. Nimble organizations can redirect their actions if a path to improvement proves to be a dead end. Absent evaluation, you are bowling with a sheet drawn across the pins.

Wrong calls still are possible, but these seven steps offer a self-correcting system for decision-making, so trash trucks won’t always be mistaken for earthquakes; or vice versa.

The preceding article was published in PA Times Online on October 1, 2013. To see the article in its original context, visit http://patimes.org/past-huh-not-so-straight-line-data-decisions/