Category: Product

Last year, I visited Melbourne and led a diner discovery and search master class for many of the city’s best restaurants.

“When your team goes home at the end of the day and they’re describing what they’re doing to their spouses or significant others or roommates, they should be using the language in the user story, because they’re inspired by the vision you’ve talked about,” Skorpen said. “If people go home, and they’re like, ‘Yeah, I can’t really explain it,’ that’s a sign the product person has failed.”

…

“One thing you learn in product management really quickly is that your instinct, your precious product idea, is going to be wrong all the time,” Skorpen said.

“9 STRATEGIES FOR WRITING MORE EFFECTIVE USER STORIES” by Tatum Hunter, based on interviews with Lauren Hutton, Vanessa Kafka, and yours truly.

Uber drivers average a 4.8 rating.

Lyft drivers a 4.86.

Average Yelp restaurant reviews are 3.71

What’s the difference?

Is the average Uber driver delivery a better experience than the average restaurant on Yelp? I don’t think so. Is Uber deleting bad reviews? I don’t think so.

Part of this is selection bias. Uber and Lyft police their platforms in a way Yelp can’t. Bad drivers get kicked out. But I think that’s only part of the story, based on my own experience managing a system with many tens of millions of reviews.

My hypothesis: People rate versus expectations. Most experiences, Uber ride, meal, whatever, meet expectations. And for Americans, meeting expectations means you get five stars.

If you’re consistently not meeting expectations, you’re going to be kicked out of the platform or you’ll go out of business (or clean up your act, in which case your average rating should improve).

So, how does this explain the difference between Yelp and Uber? Uber knows whenever you take a ride and prompts you to review. The share of rides that generate a rating on Uber is probably very high. If most people leave reviews, and the typical experience is a five star experience, most reviews on the system will be five star.

It’s very different for Yelp. Unlike Uber or Lyft or OpenTable, Yelp typically doesn’t know when you’ve been to a restaurant. They can’t prompt you to review your typical five-star meal. Only a tiny fraction of dining events are reviewed on Yelp, and for many people it’s going to take something exceptional to get them to seek out Yelp and choosing to leave a comment.

An exceptionally good experience can’t get better than a 5-star rating. There’s a lot more room to move down. Hence lower average ratings.

Originally on Twitter.

OpenTable just wrapped up Q2 goal planning. Every time we go through one of these, I refine my approach to goal setting, and wanted to codify some of my thoughts.

I’m a big believer in OKRs (aka “Objectives and Key Results”). They’re originally from High Output Management by Andy Grove, and were largely popularized in the tech sector by Google.

Writing a good OKR is hard

My standard rules:

Objective

• The objective should be aspirational, and be something that can get people excited. For search, for example, one of my objectives is “make deploys, indexing, ranking, and autocomplete crazy fast and stable.”
• For teams, I think many good objectives can stay the same quarter over quarter, serving as the core mission of the team.

Key results

• Measurable (numbers, not “crazy fast”).
• A spectrum — not boolean (e.g., “launch X” is just succeed/fail), because you want to be able to measure progress over the course of the quarter.
• Aggressive. Google’s standard is to hit a 0.6 average in grading OKRs. People who are new to OKRs are often uncomfortable with this, but managers need to push hard on this.
• Intelligible. Key results are a valuable communication tool to engineering/design teams and to counterparts throughout the rest of the company. People need to be able to understand what you’re measuring.
• Time boxed. We do quarterly planning, so our goals are typically time boxed to the quarter. For some big new launches (that might take more than a quarter of work) I’m okay with doing goals a quarter in advance, but this should be rare.

Key Initiatives

• Key initiatives are not part of classical OKRs
• For product managers who are working with engineering teams on a day-to-day basis, I think it’s also important to highlight big projects, because these big projects are a good opportunity to align with other teams and highlight complicated/impressive projects the team aims to work on.
• Initiatives are the PM or team’s hypothesis on what they’re going to do to meet the key results they’ve outlined. It’s really critical that key initiatives are hypothesis though — teams succeed or fail based on their key results, not on whether they deliver specific products.

OpenTable does OKRs by product development team, led by product managers. This means that my team has their own set of OKRs, but I think we’re missing out on OKRs that aren’t always tied to the business and on OKRs for other functions (like individual engineers).

Communication of the OKRs is just as important as writing them

We generally set aside a week at the end of every quarter for a series of death meetings to review every single team’s set of goals. It is striking how variable the approaches are for different teams.

I think the best goal meetings:

• Has a POV, and is making the case for why the work of this team matters in a macro sense
• Tell a story about what was accomplished, what will be accomplished, and how they’re related to the team’s core mission — and makes the delivering team the hero.
• Explains some of the hardest problems the team has faced and is trying to solve in simple terms. I work with our data science team, and it’s particularly important for them — even a highly technical CTO won’t know what ensembles and mixture of experts are in the context of our business.

Finally, I tried something new this planning process which I’ll be sure to include in the future. A demo. In retrospect, this seems obvious — every good planning or mid-sprint check-in should have demos, so why shouldn’t an end-of-quarter? — but I think it’s easy to get too focused on the stats. Product companies should show how they’re building products.

Product managers are in the business of communication, so concise and cogent writing is one of the most powerful tools available to us. It’s a hard muscle to develop, requiring consistent feedback and deliberate practice.

Business writing is a core part of the curriculum at the consulting firm McKinsey. Managers pore over every word written by new analysts, setting a high quality bar and demanding frequent revisions. Most companies don’t invest as much energy into training people how to write, so we’re left to figure it out on our own. It’s not easy.

One place to start is Barbara Minto’s Pyramid Principal. I was introduced to Barbara’s work by Harrison Metal and it was immediately familiar: She helped codify McKinsey’s writing style.

An easier place to start might be Julie Zhuo’s recent article on “Escaping Email Hell.” Julie offers specific changes to writing style which improve emails — and most of these tips could be applied to any kind of writing. Her best advice is to “know your end before you start.” Great writing should be a condensation of your thought — not every step in the winding journey you took.

Other books I’d recommend:

• On Writing, Stephen King
• Writing Tools, Roy Peter Clark
• The Writing Life, Anne Dillard
• The Elements of Style, William Strunk Jr. and E. B. White
• On Writing Well, William Zinsser