Reviews on marketplaces

Uber drivers average a 4.8 rating.

Lyft drivers a 4.86.

Average Yelp restaurant reviews are 3.71

What’s the difference?

Is the average Uber driver delivery a better experience than the average restaurant on Yelp? I don’t think so. Is Uber deleting bad reviews? I don’t think so.

Part of this is selection bias. Uber and Lyft police their platforms in a way Yelp can’t. Bad drivers get kicked out. But I think that’s only part of the story, based on my own experience managing a system with many tens of millions of reviews.

My hypothesis: People rate versus expectations. Most experiences, Uber ride, meal, whatever, meet expectations. And for Americans, meeting expectations means you get five stars.

If you’re consistently not meeting expectations, you’re going to be kicked out of the platform or you’ll go out of business (or clean up your act, in which case your average rating should improve).

So, how does this explain the difference between Yelp and Uber? Uber knows whenever you take a ride and prompts you to review. The share of rides that generate a rating on Uber is probably very high. If most people leave reviews, and the typical experience is a five star experience, most reviews on the system will be five star.

It’s very different for Yelp. Unlike Uber or Lyft or OpenTable, Yelp typically doesn’t know when you’ve been to a restaurant. They can’t prompt you to review your typical five-star meal. Only a tiny fraction of dining events are reviewed on Yelp, and for many people it’s going to take something exceptional to get them to seek out Yelp and choosing to leave a comment.

An exceptionally good experience can’t get better than a 5-star rating. There’s a lot more room to move down. Hence lower average ratings.

Originally on Twitter.



OpenTable’s product book club recently tackled Marty Cagan’s Inspired (which has a new edition out recently). My first week at OpenTable was a workshop for our product development organization led by Cagan, and re-reading the book was a great reminder of how significantly he’s defined my product management philosophy.

My take-aways for great product teams:

  • Product teams, with a focused mandate and appropriate staffing
  • Process matters enormously: Good process isn’t just a marginal competitive advantage
    • Good process involves launching and iterating fast with a dual-track discovery/development process
    • Good process gets the people building the product involved in figuring out what to build ASAP
    • Good process means delegating decision making authority down to individual PMs; decision makers who are close to the customer are a competitive advantage
  • Encapsulate business logic so that teams can own across the broader product


OpenTable just wrapped up Q2 goal planning. Every time we go through one of these, I refine my approach to goal setting, and wanted to codify some of my thoughts.

I’m a big believer in OKRs (aka “Objectives and Key Results”). They’re originally from High Output Management by Andy Grove, and were largely popularized in the tech sector by Google.

Writing a good OKR is hard

My standard rules:


  • The objective should be aspirational, and be something that can get people excited. For search, for example, one of my objectives is “make deploys, indexing, ranking, and autocomplete crazy fast and stable.”
  • For teams, I think many good objectives can stay the same quarter over quarter, serving as the core mission of the team.

Key results

  • Measurable (numbers, not “crazy fast”).
  • A spectrum — not boolean (e.g., “launch X” is just succeed/fail), because you want to be able to measure progress over the course of the quarter.
  • Aggressive. Google’s standard is to hit a 0.6 average in grading OKRs. People who are new to OKRs are often uncomfortable with this, but managers need to push hard on this.
  • Intelligible. Key results are a valuable communication tool to engineering/design teams and to counterparts throughout the rest of the company. People need to be able to understand what you’re measuring.
  • Time boxed. We do quarterly planning, so our goals are typically time boxed to the quarter. For some big new launches (that might take more than a quarter of work) I’m okay with doing goals a quarter in advance, but this should be rare.

Key Initiatives

  • Key initiatives are not part of classical OKRs
  • For product managers who are working with engineering teams on a day-to-day basis, I think it’s also important to highlight big projects, because these big projects are a good opportunity to align with other teams and highlight complicated/impressive projects the team aims to work on.
  • Initiatives are the PM or team’s hypothesis on what they’re going to do to meet the key results they’ve outlined. It’s really critical that key initiatives are hypothesis though — teams succeed or fail based on their key results, not on whether they deliver specific products.

OpenTable does OKRs by product development team, led by product managers. This means that my team has their own set of OKRs, but I think we’re missing out on OKRs that aren’t always tied to the business and on OKRs for other functions (like individual engineers).

Communication of the OKRs is just as important as writing them

We generally set aside a week at the end of every quarter for a series of death meetings to review every single team’s set of goals. It is striking how variable the approaches are for different teams.

I think the best goal meetings:

  • Has a POV, and is making the case for why the work of this team matters in a macro sense
  • Tell a story about what was accomplished, what will be accomplished, and how they’re related to the team’s core mission — and makes the delivering team the hero.
  • Explains some of the hardest problems the team has faced and is trying to solve in simple terms. I work with our data science team, and it’s particularly important for them — even a highly technical CTO won’t know what ensembles and mixture of experts are in the context of our business.

Finally, I tried something new this planning process which I’ll be sure to include in the future. A demo. In retrospect, this seems obvious — every good planning or mid-sprint check-in should have demos, so why shouldn’t an end-of-quarter? — but I think it’s easy to get too focused on the stats. Product companies should show how they’re building products.



Product managers are in the business of communication, so concise and cogent writing is one of the most powerful tools available to us. It’s a hard muscle to develop, requiring consistent feedback and deliberate practice.

Business writing is a core part of the curriculum at the consulting firm McKinsey. Managers pore over every word written by new analysts, setting a high quality bar and demanding frequent revisions. Most companies don’t invest as much energy into training people how to write, so we’re left to figure it out on our own. It’s not easy.

One place to start is Barbara Minto’s Pyramid Principal. I was introduced to Barbara’s work by Harrison Metal and it was immediately familiar: She helped codify McKinsey’s writing style.

An easier place to start might be Julie Zhuo’s recent article on “Escaping Email Hell.” Julie offers specific changes to writing style which improve emails — and most of these tips could be applied to any kind of writing. Her best advice is to “know your end before you start.” Great writing should be a condensation of your thought — not every step in the winding journey you took.

Other books I’d recommend:



Rewards vs. Culture

When I was in college, I spent two years as editor in chief of our daily online-only newspaper, The Daily Gazette. One of my first projects was leading a redesign of our website — and that included adding advertising; overnight, we were pulling in a thousand dollars a month.

I didn’t really know what to do with the money (the college paid our costs), so I came up with a scheme to divvy up the money as a salary of sorts to our team. Each semester, we’d pay writers, photographers, and editors a few hundred dollars — but they had to be active participants, meeting a quota of contributions. I imagined this could lead to a golden age for quality content.

Instead, the program failed miserably. Story contributions and staff attendance dropped rapidly over the course of the semester.

Next semester, we changed course. We stopped paying the team (apart from a couple, more significant, work-study scholarships) and instead used the money to buy t-shirts, send people to conferences, hire a journalism coach, and throw parties. This became a turning point for the Gazette: Our staff grew, the team wrote a lot of great stories, and we were named one of the top three college news sites in the country.

Looking back, I now realize that this challenge is a great lesson in the difference between rewards and culture.

What I realized after that first semester is that when we started paying our staff, working for the Gazette became a job — not a passion hobby. Our staff had tons of other opportunities, and a couple hundred bucks wasn’t worth hours of their life. When we pivoted how we used the money, though, we built tight relationships within the team, we created a sense of camaraderie, and we made the Gazette a social touch point at the school.

People want to get paid appropriately, of course, but as people move up Maslow’s hierarchy they want more than just a paycheck. I’m always thinking back to my time at the Gazette, and wondering how I can focus on building a culture so that work isn’t just an obligation.

So what should you do? Events like OpenTable’s recent internal hackathon are great. Share books with colleagues, so that you establish common frameworks and mental models. Print t-shirts, host board game nights, sponsor wine tastings. And don’t give undervalued rewards.