Covering Wicked Problems

I think every writer, every journalist, every scholar, should tell you where he’s coming from before he tells you what he knows. I am not a science journalist, or a science blogger, or a scientist who writes. But I am interested in your world, and I try to follow developments in it. My field of study is what I call “pressthink,” which is sort of like groupthink– but for people in journalism. Lately I have been fixated on the problems of the press as it tries to adapt to the digital world. So that’s what I do. But it’s not where I’m coming from.

Culturally, I’m a secular Jew. (From New York.) Demographically, a baby boomer. Socially, I’m an introvert who has learned to fake conviviality. Politically, a liberal democrat. Musically: lost. Intellectually, I am a pragmatist. Among professional philosophers, practitioners of what used to be called “moral science,” pragmatism is sometimes called the only homegrown American philosophy. William James and John Dewey are the heroes of the discipline, and two big ideas animate us. First: the test of a good idea is what you can do with it. A thinker should try to be useful. Second: pragmatists believe that our knowledge advances not when we have the best theory, or the best data, or the best lab, but when we have really good problems.

And that’s what I have for you today: a really juicy puzzle. It begins with a distinction that I have found useful. The distinction is between tame and wicked problems. Now given what’s happened to science writer Jonah Lehrer lately I should tell you that I’ve written about this issue before and since I said it about as well as I could say it then, I am going to say it in a similar way again… okay?

Here is a problem that anyone who has lived in New York City must wonder about: it’s impossible to get a cab at 5 pm. The cause is not a mystery: taxi drivers tend to change shifts around 4 to 5 pm. Too many cabs are headed to garages in Queens because when a taxi is operated by two drivers 24 hours a day, a fair division of shifts is to switch over at 5 o’clock. Now this is a problem for the city’s Taxi and Limousine Commission, it may even be a hard one to solve, but it is not a wicked problem. For one thing, it’s easy to describe, as I just showed you. That right there boots it from the category.

Wicked problems have these features: It is hard to say what the problem is, to define it clearly or to tell where it stops and starts. There is no “right” way to view the problem, no definitive formulation. There are many stakeholders, all with their own frames, which they tend to see as exclusively correct. Ask what the problem is and you will get a different answer from each. Someone can always say that the problem is just a symptom of another problem and that someone will not be wrong. The problem is inter-connected to a lot of other problems; pulling them apart is almost impossible. In a word: it’s a mess.

But it gets worse. Every wicked problem is unique, so in a sense there is no prior art and solving one won’t help you with the others. No one has “the right to be wrong,” meaning enough legitimacy and stakeholder support to try things that will almost certainly fail, at first. Instead failure is savaged, and the trier is deemed unsuitable for another try. The problem keeps changing on us. It is never definitely resolved. Instead, we just run out of patience, or time, or money, or political will. It’s not possible to understand the problem first, then solve it. Rather, attempts to solve it reveal further dimensions of the problem. (Which is the secret of success for people who are “good” at wicked problems.)

Know any problems like that? Of course you do. Climate change! What could be more inter-connected than it? How the hell do we define it? Is it the burning of fossil fuels? Is it modernization? Capitalism? Externalities? The whole system of states? Man’s false dominion over nature? Someone can always say that climate change is just a symptom of another problem– our entire way of life, maybe — and he or she would not be wrong. We’ve never solved anything like it before, so there’s no prior art. Stakeholders: everyone on the planet, every nation, every company. Super-wicked, as some have put it.

In the United States, almost everyone knows that our public schools don’t work. Almost no one admits that this too is a wicked problem.

Tame problems are not easy to solve, but they are easy to define, to fix in a proper frame. How to build a bridge over the Mississippi that won’t fall down, and determining what it will cost: we’ve tamed that one. The engineers apply the science and select the best design within the constraints the government has put forward. The politicians figure out how to pay for it and how to sell it. We can know in advance what kind of expertise will be needed.

Wicked problems aren’t like that. As the founders of the concept said in 1973, “You don’t understand the problem until you have a solution.” When I first moved to New York City in 1980, lowering the sky high crime rate was a wicked problem. Now we know that the same people who were evading the fare in the subways were committing more violent crimes; if you arrest them for these petty violations the bigger violations decrease. And the crime rate in New York City has fallen dramatically since 1980.

It’s not that simple, of course. Other factors are involved and there are unintended consequences. In New York these include a stop-and-frisk practice that may be unconstitutional because it is so weighted toward minorities. That’s the way wicked problems roll. Lots of times you never solve them. Each successful action reveals a further dimension of the problem. Which is frustrating. But less frustrating if we learn to distinguish between wicked problems and the other kind.

Remember: we’re all pragmatists here today. So instead of shooting holes in my description of wicked problems (which you certainly could do…) try asking yourself: what is useful about this distinction? What is the intellectual work–or as Williams James always said, the “cash value”–of the concept? I have a short answer and a longer one.

On April 15, 2010 Nick Clegg, David Cameron and Gordon Brown took the stage for a candidates debate. Imagine if the questioning had been divided into “current issues” and… “wicked problems.” Wouldn’t that feel fresh and different? More… grown-up?

That’s my short answer. Now for my longer answer. How to cover the wicked ones differently from the tame problems is my idea of a juicy puzzle in press practice. If we could solve it, our knowledge would be greatly advanced. So now I want to sketch for you what a wicked problems “beat” might look like.

Suppose we had such a beat. How would you do it? How might it work? I have ten descriptors to share with you. Ten ways of imagining how a wicked problem beat would operate.

1. It would be a network, not a person. My friend Dan Gillmor, the first newspaper journalist to have a blog, said something extremely important in 1999, when he was reporting on Silicon Valley for the San Jose Mercury News. “My readers know more than I do.” So simple, and profound. Any beat where the important knowledge is widely distributed should be imagined from the beginning as a network.

The wicked problems beat would have to be a network because the people who know about coping with such problems are unevenly distributed around the world. Imagine a beat that lives on the Net and is managed by an individual journalist but “owned” by the thousands who contribute to it. Journalists from news organizations all over the world can tap into it and develop stories out of it, but the beat itself resides in the network. In the way I imagine this working, news organizations that are members of the beat might “refer” problems discovered on other beats to the wicked problems network and say… “look into this, will you?” The beat would in turn refer story ideas and investigations back to the member newsrooms.

2. The beat would be pattern-based. Meaning: all instances of the pattern, no matter where they are found, are fair game for the beat. Wicked problems turn up in business settings all the time, but this is not a business beat. Wicked problems originally emerged from urban planning and design. But this is not a public policy beat. You can find wicked problems in politics, education, criminal justice, economic development and of course the environment. So this is a beat that would cut across newsroom verticals, looking for places where people get stuck because they are treating a wicked problem as if it were tame.

3. A classic narrative stands at the heart of the beat. How many of you have heard of “agile development” in software? It’s an approach to doing big software projects that grew out of the recognition that many big software projects are in fact wicked problems, and that the existing approach–called the waterfall model–frequently ended in massive failure. In the prior model, the software team would survey the clients needs, develop a feature list for what the software had to do, prioritize that list, get a check off and a budget from the client, then determine the best design, build it, test it, de-bug it, and deploy it. The client would then say: this is not what we asked for! This sucks! And the developers would (try to) say back: what are you talking about? This is exactly what you asked for! We don’t suck, you suck!

Why does this happen? Because software clients are stupid and they don’t know what they want? Nope. It’s what I said earlier. “You don’t understand the problem until you have a solution.” We might call this the founding insight of the wicked problems school of thought. So a better way to go about these huge software projects is to listen to the clients and quickly prototype something that solves one little problem. Then show it to the client, knowing that it will probably reveal further aspects of the “big” problem the software has to solve. Then you do that again. And gradually you get there. (Of course, I am simplifying–perhaps over-simplifying–a rather contentious term.)

Jumping back and forth from a global understanding that is constantly in revision to local solutions that are constantly being tested: this is a better way to go. Better than: gather information, outline the options, analyze costs, pick the best option, hire the experts, and implement. Agile development is learned behavior for coping with wicked problems. Whenever something like that happens, the wicked problems beat springs into action. Because that’s a great story. It is in fact the classic story on this beat: getting stuck, and then getting unstuck.

4. The beat would be global because wicked problems are a global phenomenon. They are found everywhere. In fact, a good funder for the beat would be the World Bank. Come to think of it, I gave a talk to their communication officers a few years ago. I think I’ll suggest it.

5. The wicked problems beat can’t rely on the experts. Wicked problems are in a way a deep reflection on the limits of professional expertise. No matter how good our software designers are, they are still going to produce big projects that clients hate because the problems are not technical. They are matters of judgment. This is another reason the beat has to be a network, and it has to be global. It can’t rely on authorized knowers and institutional experts.

6. The “stars” of the beat would be people all over the world who seem to be good at wicked problems. And the firms that listen to those people. Meaning: those who show good judgment when all the options can never be on the table because (once again) we don’t understand the problem until we’ve solved it. The bread and butter of the beat is the profile of those who “get” that.

7. The beat would treat denial as a news story. I’m 56 years old. The older I get, the bigger denial looms as a wild card in human affairs. I see it everywhere today. A wicked problems beat would have to be especially well tuned to denial, which is the stage that precedes that classic moment when participants realize they have a wicked problem on their hands. Denial is a psychological category, so it is an inherently risky thing to report upon. After all, if I disagree with your description you can always say: Jay, you’re in denial.

But I think it can be done. And those of you who have followed the climate change story understand that it must be done. The starting point has been pointed out to us by Chris Mooney. The more educated and intelligent the denialist is, the more intractable the problem seems to be. People become expert in their own systems for ignoring reality. Systems become expert in concealing from their operators wicked problems. That is something we can learn to report on.

8. The wicked problems beat would have to be a learning machine. We know that reporters get smarter about a beat the longer they do it, but what if the beat itself got smarter? Wouldn’t that be cool? That’s how I envision the wicked problems beat. Over time, it gets smarter about locating the stories that help us cope with wicked problems. Not solve: cope.

In one of the articles I read for this talk, a Harvard Business Review writer was talking about companies that do wicked well. He said they “continuously scan the environment for weak signals rather than conduct periodic analysis of the business landscape.” A networked beat should do that. Over time, it should get better and better at picking up the faint signs of wickedness in problems that do not yield to expertise or that overwhelm their would-be solvers with complexity. Obviously the global financial system is rife with such. But what we need is an alert system that works before the crash. After all, the press is supposed to inform us, yes. But it is supposed to inform us in time to make a difference.

9. The beat would have a goal, a mission. And I think I can say what it is. Earlier, I said it is characteristic of wicked problems that key stakeholders define the problem differently. That plurality of frames is inevitable. But what’s not inevitable is the stakeholders’ mutual ignorance of each other’s incompatible starting points. There is no kumbaya moment. You never get everyone on the same page. Consensus? You must be kidding. In dealing with wicked problems these are vain hopes, signs of the stupid. What’s possible is a world where different stakeholders “get” that the world looks different to people who hold different stakes. In reporting on wicked problems, that’s the goal. That and learning from the work of people who do wicked well.

Which brings me to something I’ve wanted to say for a long time. Journalism belongs to the vernacular, or it has no place in the world. When reporters render things, they have to find a common language for them. That is why the great vice in journalistic writing is cliché. Cliché is the vernacular in its spent state. Savage clarity is the vernacular coming alive again. Rendering stakeholders intelligible to one another, and to the larger public, is journalism at its best. No one should become a journalist who cannot tolerate the paradox of being a specialist… in the vernacular.

10. The wicked problems beat implies a view from somewhere. I said at the beginning of this talk that every journalist, every writer should tell us where he’s coming from. So it is with networked beats. So it is with this beat. The wicked problems beat is not a View from Nowhere thing. It starts from the limits of professional expertise. It is a reflection on unmanageable complexity. It preaches humility to the authorized knowers. It mocks the one best answer and single issue people. It seeks to deliver us from denial.

I am coming in for landing. There has to be someone in this crowd today who is sitting there thinking: Hey, that’s nifty, this new beat you’re sketching for us, but realistically who’s going to pay for it? We can’t even cover the tame problems with the shrinking work force we have in most newsrooms. We can’t even cover science. Now you’re dumping on us this whole other job?

Well, I have to confess. I don’t have a solution for that. And you know why? Because how to fund and sustain a vigorous public service press is not just a really hard problem. It’s a wicked one! Of course it is. So we better get smart about such things. Before it’s too late.

Thank you very much.