Book Review: Wisdom of Crowds

I liked this book so much I wrote a book review originally published on iMedia in the Fall of 2005. iMedia is the “Ad Age” of Internet marketing so it was written with a marketer’s point of view. You can get the book here. Though I read and reviewed this about a year and a half ago, the notion of “collective intelligence” has risen in prominence quite a bit since then so thought it was worth sharing again.

If you look beneath the surface of some successful companies, you'll see they employ strategies espoused in James Surowiecki's book. A great example is a company run by my friend Ian Morris -- the CEO of HouseValues. There's much more than meets the eye that drives their success. The full title of the book is "The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations," by James Surowiecki.

Surowiecki aims to be the next New Yorker contributor to have a mass appeal book, ala Malcolm Gladwell. He clearly wants to position this book as the next "The Tipping Point" -- combining cognitive science and other disciplines into a book addressing business, politics, society and economies. The book’s relevance to marketing may not be as obvious as The Tipping Point, although there are examples from our industry. As Surowiecki states, “Google is built on the wisdom of crowds. The core of the system is the PageRank algorithm -- a calculating method -- that attempts to let all the Web pages on the Internet decide which pages are most relevant to a particular search.” He goes on to say “With most things, the average is mediocrity. With decision making, it's often excellence. The idea of the wisdom of crowds isn't that a group will always give you the right answer but that on average it will consistently come up with a better answer than any individual could provide.”

While the stakes in marketing may not be as high as space shuttle missions or stock markets (two of Surowiecki's other examples), there are lessons in this book that marketers could apply in a variety of ways. Whether you're making business strategy decisions or developing a marketing campaign, many of us have seen the ill effects of "groupthink" versus bringing together diverse groups within an effective framework. Fortunately, the Internet can enable the elements of a "wise crowd."

This book made me think about how collective wisdom could affect decisions such as ad campaign development (creative decisions, media buying, etc.), product development decisions, industry standards (surely there's a way to move things forward more rapidly) and many other decisions made within your company or across teams representing clients, agencies and technology providers. From my own experience working with technology companies targeting the marketing community, I can see many ways to apply principles in this book to product development, sales and marketing.

Corporations and industry bodies have generally been unwilling to improve their decision making by tapping the collective wisdom of their employees and members. Those who can harness the potential for wisdom that exists within crowds of people will have the world as their oyster.

Perhaps the most significant point for readers isn’t the perspective Surowiecki provides to the inward-facing, organizational structure/behavior perspective, but rather the outward-facing: how marketers can better understand their customers -- how they think, why they think that way, and how their ability to communicate with each other (rather than just with customer service and technical support) raises their collective IQ.

It could be said that the wisest crowd out there is the billions-big horde of Internet users, who consistently use the Web in smart ways that neither technologists nor marketers would have dreamt up. The companies and marketers who grasp the implications of this will in turn develop products and services more in tune with their customers’ needs, as well as have the accompanying Web sites and marketing campaigns to harness these insights.

While our society often trusts experts and distrusts the wisdom of the masses, Surowiecki argues that "under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them." He uses a variety of examples ranging from simple challenges such as a crowd guessing the weight of an ox to incredibly complex: another crowd located a lost submarine where the best approximation was 20 miles wide and thousands of feet deep. It was eventually found 200 yards from where the group estimated it would be. This despite the fact that no one knew why the sub sank, no one had any idea how fast it was traveling or how steeply it fell to the ocean floor. Other compelling examples include how SARS was solved, and a method for predicting election outcomes with great accuracy.

The author outlines four elements required to have a wise crowd:

• Diversity of opinion: Each person should have private information even if it's just an eccentric interpretation of the known facts. 
• Independence: People's opinions aren't determined by the opinions of those around them.
• Decentralization: People are able to specialize and draw on local knowledge.
• Aggregation: Some mechanism exists for turning private judgments into a collective decision.

He also cites examples of groups where these elements are missing with sometimes disastrous consequences. Small groups can make very bad decisions because influence is more direct and immediate and small-group judgments tend to be more volatile and extreme. Large groups missing the four elements can also have disastrous results.

One significant example that Surowiecki describes concerns the Space Shuttle Columbia's Mission Management Team (MMT). The team violated nearly every rule of good group decision making. As Surowiecki outlined, “the team's discussions were simultaneously too structured and not structured enough. They were too structured because most of the discussions -- not just about the debris strike, but about everything -- consisted of the MMT leader asking a question and someone else answering it. They were not structured enough because no effort was made to ask other team members to comment on particular questions. This is almost always a mistake, because it means that decisions are made based on a very limited supply of analysis and information.”

One of the consistent findings from decades of small group research is that group deliberations are more successful when they have a clear agenda and when leaders take an active role in making sure everyone gets a chance to speak. In small groups, diversity of opinion is the single best guarantee that the group will reap benefits from face-to-face discussion.

Conversely, in a stock market bubble all the conditions that make groups intelligent -- independence, diversity, private judgment -- disappear. Whether it was the dot-com bust or the run-up of bowling stocks 40 years ago, stock markets have the potential to lose key elements that make them generally effective. As we are on the 75th anniversary of the 1929 stock crash, we need to remain conscious of the limits of the risks when all the elements of a wise crowd are absent.

The following are additional ideas from this useful book that are salient to the four elements of “wise crowds” -- Diversity, Independence, Decentralization and Aggregation:

Diversity of opinion and background

• When there is a lot of uncertainty, such as in the early days of an industry where the winners and losers haven't been sorted out, it's key to have a system that encourages, and funds, speculative ideas, even though they may have only slim possibilities of success. Even more important is diversity -- not in the sociological sense, but rather in a conceptual and cognitive sense. What makes a system successful is its ability to generate lots of losers, recognize them as such and then kill them off. Sometimes the messiest approach is the wisest.
• Diversity helps because it adds perspectives that would otherwise be absent and because it takes away, or at least weakens, some of the destructive characteristics of group decision making.
• We know that the crowds that make the best collective judgments are crowds where there's a wide range of opinions and diverse sources of information, where people's biases can cancel themselves out, rather than reinforcing each other. Individual irrationality can add up to collective rationality.
• Decision markets are well suited to companies because they circumvent the problems that obstruct the flow of information at too many firms: political infighting, sycophancy, and a confusion of status with knowledge. The anonymity of the markets and the fact that they yield a relatively clear solution, while giving individuals an unmistakable incentive to uncover and act on good information, means their potential value is genuinely hard to overestimate.
• Studies have found that groups of smart and not-so-smart people almost always do better in decision making than a group just of smart people. The development of knowledge may depend on maintaining an influx of the naïve and ignorant, because competitive victory does not reliably go to the properly educated. My take-away: Teams I've worked on always benefit from the fresh perspective of a newcomer.
• Homogenous groups are great at doing what they do well, but they become progressively less able to investigate alternatives. It also fosters the palpable pressures toward conformity that groups often bring to bear on their members.
• Diversity contributes not just by adding different perspectives to the group but also by making it easier for individuals to say what they really think.

Independence

• Paradoxically, the best way for a group to be smart is for each person in it to think and act as independently as possible.
• Independence doesn’t mean isolation but it does mean relative freedom from the influence of others.
• Independence is critical for two reasons 1) it keeps mistakes that people make from becoming correlated; 2) independent individuals are more likely to have new information rather than the same old data everyone is familiar with.
• If you want to improve an organization's or economy's decision making, one of the best things you can do is make sure, as much as possible, that decisions are made simultaneously rather than one after another.

Decentralization

• What do we mean by “decentralization?” Power does not reside in one central location, and many of the important decisions are made by individuals based on their own local and specific knowledge, rather than by an omniscient or farseeing planner.
• Decentralization’s greatest strength is that it encourages independence and specialization on the one hand, while still allowing people to coordinate their activities and solve difficult problems on the other.
• A decentralized system can only produce genuinely intelligent results if there's a means of aggregating the information of everyone in the system.

Aggregation

• Groups generally need rules to maintain order and coherence, and when those elements are missing or malfunctioning the result is trouble. Groups benefit from talking to and listening to each other, but, paradoxically, too much communication can make the group as a whole less intelligent.