Search, discovery and marketing

One of the clearest places to see this problem of ‘too much’ is Yelp. I’ve been fascinated by how many companies are effectively trying to unbundle Yelp, despite that fact that (unlike Craigslist) it’s a modern technology company that does most of the things one would expect it to. But where people unbundling Craigslist generally try to peel off a category and deliver a modern experience, the people going after restaurant listings are often doing so with constraint. That is, instead of giving you every single restaurant that’s within 2 miles and that lots of people liked, they give you 10 restaurants. … People are attacking crowdsourced universal scale with constraint, curation and personal preference.

Looking at these companies, it strikes me that actually, saying that ‘Yahoo’s directory didn’t scale’ misses the point. What we’re really seeing is a trade-off between two problems. You can have a list, solving discovery and recommendation, but once the domain gets big then your list is either unusably long or partial and incomplete (and perhaps uneconomic to maintain). Or you can have a searchable index of everything but you’re on your own working what’s good and finding things you didn’t know to search for.

Search, discovery and marketing from Benedict Evans

Curation also works best in places where taste matters — the “best” book, the “best” restaurant. An algorithm can’t provide social validation beyond “the most popular.” What’s interesting about Yelp is that their focus on user generated content  means it’d be hard for them to add sources of curation outside of their users — there’s no list of restaurants with Michelin stars, or James Beard winners, and there probably can’t be one without undermining the core conceit of the platform.

Killing the Creative Class

In Culture Crash: The Killing of the Creative Class, Scott Timberg explores the convergence of forces which have combined to “eviscerate the creative class” — those people who make, produce, review, and sell culture; he includes everyone from record store clerks to authors.

While I sympathize with Timberg, and share some of his nostalgia for the late 20th century, I deeply disagree with his assessment of what we’ve lost, and how we lost it. At a high level, Timberg blames the rise of the internet for the decline of the creative class, through piracy, disintermediation, and mass retail.

His piracy arguments feel tired; the point has been argued elsewhere (see this, this, this, and this). While counterintuititve, there isn’t much evidence to suggest that modern piracy substantially harms most artists; the rise of high-quality distribution services massively undercut piracy (see: Spotify).

On disintermediation, his description of record, book, and movie store clerks is striking. They were “men who’d given up something else do [this work] out of love for the music.” He describes they as having a moral commitment. It is as if cultured clerks are modern-day monks, dedicated to art instead of God.

But the decline of Western civilization’s long-time keeper, the Catholic Church (and with it, monks & monastaries) no more sounded a death-knell for our culture than the rise of the Internet over the past three decades has.

In particular, Timberg fails to place the creative class he mourns in the context of history: Mass-produced culture is a new phenomena. There were few book clerks before Gutenberg. Motzart and Michelangelo produced culture with funding from noble patrons, not from a creative middle-class. Art and culture will need to change in response to new technologies and social mores, but it will survive.

I suspect we will see a shrinking professional creative class, but a flowering of amateurs. I’m not concerned about the production of culture, but I’m less clear on how culture is discovered. Timberg suggests that the canon of classics is undersiege, and its hard to disagree. There are fewer “expert” voices saying what should be consumed, and its harder to find the signal in the noise.

This isn’t necessarily negative. Old cultural gatekeepers often enforced mass culture — disadvantaged or fringe artists could be locked out. But I find it hard to believe that The New York Times Bestseller List is the best way to identify our cultural canon.

In part, this is because I think why something is recommended is just as important as what was recommended.

For many pieces of culture, the story and recommendation is a critical part of providing value. What is the story of this piece? How does it fit into the larger movement? What is its place in history? Am I supposed to like it? I suppose I believe that culture benefits from a hint of elitism. And I’m not sure that algorithms can provide an elite judgement anytime soon.

For many, the consumption of culture isn’t too different from the consumption of wine: We take pleasure, backstory, and price — in addition to the culture itself.

Computer algorithms can provide recommendations (as shown by Netflix), and will improve. But the algorithms that undergird machine-learning recommendations work by finding counterintuitive and subtle links between people and the culture recommended — I don’t believe exposing these recommendations would be compelling to most users.

Some companies have two separate algorithms: one to produce a recommendation, and one to produce a compelling set of reasons. This should eventually produce compelling rationales (assuming “success” is properly defined and fed into the algorithm — easier said than done), but if people don’t have faith in the algorithm, it can’t impart greater meaning and value to pieces of culture.

So I’m not sure how we can protect this judgement.

I don’t think recommendations do well in a winner-take-all marketplace, and instititutions like Michelin or Pitchfork are struggling to remain viable as businesses.

In part, these publications and cultural touchstones are being replaced by user generated content [UGC]. But UGC-centric sites struggle with authority — is a lengthy review from a Yelp elite reviewer more important to Yelp than a review from Michelin? Should a university professor and expert in a subject have more power than a long-time Wikipedia editor to edit Wikipedia?

The issue might not be insurmountable — Stack Overflow seems to have resolved some of these issues, and FourSquare is playing with interesting mesasures of “expertise” — but there are no comparable cultural organizations.

Where do we go from here? I’m not sure.