The Folly of Crowdsourcing

Before I make a leap, especially one involving travel or durable goods, I like to feel that I’m basing my decision on hard information. I used to think that the Internet held the answer to this problem, but instead it’s only raised my frustration and anxiety when I’m trying to book a scuba trip or decide if a particular book is worthy of my attention. This problem came into slightly better focus for me last Tuesday, when I read two articles, back-to-back, from The Atlantic Monthly.

The first article, Wayne Curtis’s Weni, Widi, Wiki, told the story of a visit to Seattle using only Internet guides, specifically those with user-generated content. It seemed to highlight for me the reason I’m no longer so interested in Yelp, Amazon Reviews, or most of the crowdsourced Internet. My first thought after reading this article was that I was simply tired of opinions.

Several years ago, I enjoyed being a fire-breathing opinion columnist with the primary mission of thinking provocative thoughts and the secondary mission of generating page views, with only a tertiary mission of being right. If I was wrong in a way that made people stop and think about the problem, I’d won.

I don’t think I’d enjoy that job much today. By and large, I just don’t care whether people like my opinions, or agree with them. If people ask me what I think, I’m more than happy to share my opinions, but I’m not all that likely to volunteer what I think. (This has made it harder for me to blog, since I feel so reluctant to say anything without prompting.)

Then I read the very next article, Corby Kummer’s Beyond the McIntosh, about John Bunker, a man who has dedicated his career to preserving heirloom varieties of apples. At the Capitol Hill farmer’s market, I buy most of my apples from one guy, who grows a mix of things people have heard of, and things that most people haven’t. I discovered my favorite variety of apple there, and was tickled pink to discover that it’s the same as John Bunker’s favorite: the Black Oxford, native to Maine, and the apple that launched Mr. Bunker on his career.

If I didn’t care about opinions, why would I be so excited by the information that John Bunker’s favorite apple was the same as my own?

The answer, I think, is the difference between solo and aggregated opinions. Crowdsourced reviews have the same grey, mushy feel of meals at The Cheesecake Factory. When I first paged through the copy of Zagat’s restaurant guide for Seattle I’d been sent, I couldn’t believe that people were as likely to praise the Cheesecake Factory as to dis Lark, my favorite Seattle restaurant.

But of course, Zagat’s guide polls hundreds or thousands of people (myself included), and averages out their opinions. On average, people will prefer an unchallenging place, with giant portions of bland food, to a restaurant specializing in small plates of often strange or exotic food – and that’s true even if each individual’s preferences are otherwise.

I suspect that almost everyone who reviewed The Cheesecake Factory has an interesting, even unique, set of food preferences, and that I could talk for a good long while with most of them and find their individual opinions fascinating, even when I disagreed with them. But, taken together, these individual opinions cancel each other out and leave you with an uninflected average opinion.

I remember some science fiction story I read as a teenager, in which one character posited that the Venus of Willendorf represented an averaged-out map of what men desired in women, which puts me in mind of Jason Salavon’s work.

In other words, preferences are more interesting individually than when they’re aggregated. This is the lesson of Apple versus Microsoft, where the former is a distillation of one man’s particular aesthetic and the other is the product of endless usability tests. I’d go so far as to assume that Microsoft’s software would be more interesting, and more pleasing to myself, if it were the product of Bill Gates’s personal vision, or even that of a single usability research subject. But such software would likely be less successful overall, if we define success as marketplace success, the definition by which The Cheesecake Factory succeeds.

I never pick up my Zagat, and I’m not much more likely to rely on Amazon, Yelp, TripAdvisors, or its myriad crowdsourced descendants. I’d rather go up to a stranger, as my boss did one night in New York City, and ask the name of his favorite bar. In fact, one recommendation was the first bar I ever remember going to in order to see a band, The Continental. I was surprised, and more than a little pleased, to find that they were still there. But most of the recommendations were places I hadn’t heard of, or would have found on my own.

Sometimes aggregated opinions are toxic. I had a particularly bad experience when I tried to find reliable information on picking out a Blu-Ray player. HD-DVD partisans had so poisoned the pages on Amazon and other sites that I was unable to tell who had really used the player in question, and who was merely trying to sow uncertainty and push people to HD-DVD instead. Claims and counter-claims proliferated, with no end in sight.

In the end, I bought a low-end Sony player, upgraded the firmware to the latest before trying anything else, and have had not a single one of the myriad reported problems. Was I just lucky, or was all of the concern overblown? The plural of “anecdote” is not “data”, even if it’s my anecdote. What, exactly, do we call the plural of “opinion?”