Online peer reviews do sell things…up to a point
Here’s an interesting report, from our very own University of California at Berkeley, as reported in the San Francisco Business Times: “[G]ood online reviews on Yelp do indeed bring in more customers.” Specifically, “a half star rating increase (1 to 5 scale) meant a 19 percent greater likelihood that a restaurant’s seats would fill up during peak hours.”
The researchers did not have an explanation for this phenomenon (which actually has some important limits, which I’ll get to in a minute), but I do. Now, I’m one of those people who likes and depends on restaurant reviews. We have a ton of restaurants here in the San Francisco-Oakland-Berkeley area, of all types, at all price levels, from just about every ethnicity in the world. So it can be confusing and intimidating to decide on a new place to eat. Under the circumstances, I’ll often turn to two sources for recommendations: Yelp, and the San Francisco Chronicle’s great restaurant reviewer, Michael Bauer. A bunch of great Yelp reviews is enough to persuade me to try someplace out, while a single Bauer “must eat there” does the same thing.
I think that’s the reason why Yelp reviews work: people, like me, believe in peer recommendations (such as Yelp’s) and also in expert reccos (such as Michael Bauer’s). Of course, just 1 or 2 glowing peer reccos for a particular place won’t work for me (or anyone else, I should think), because they could always be from the owner’s cousin and mother. And 1 or 2 glowing reviews won’t do it at all, if they’re negated by 6 or 7 “worst experience of my life,” “would never go back there,” “AVOID AT ALL COSTS!”
But one great Michael Bauer review will send me to the joint. I guess, to my way of thinking, there is an emerging parity between expert reviews, on the one hand, and peer reviews, on the other, but that parity only works if the peer reviews (such as Yelp’s) are overwhelmingly positive. So Michael Bauer isn’t going to have to look for a new job anytime soon. When it comes to food, people still depend on restaurant critics. (At least, in a foodie town like Ess Eff.)
I mentioned above that the U.C. Berkeley study had important limits:
(1) “For restaurants with Michelin stars, for example, the Yelp reviews were irrelevant.”
(2) “Restaurants that were rated in popular guidebooks or newspaper rankings got less of a Yelp bump. They ‘did not see a statistically significant effect from the Yelp rankings,’ the economists said.”
Let’s take (2) first. This just confirms my own reasoning: I’ll take Michael Bauer over Yelp 95% of the time. Even if there were positive Yelp reviews, one critical Bauer review canceled them out. Call me old-fashioned, but I still believe experience counts over simple enthusiasm (such as the type you see on Yelp and, for that matter, on “Check, Please!).
As for (1), my hunch is that the kind of people who review restaurants on Yelp probably don’t frequent Michelin restaurants. Why not? They’re too expensive; the people who eat at French Laundry, Coi and Benu are not likely to post their experiences on Yelp, and the people who are considering eating at French Laundry, Coi and Benu are not turning to Yelp for advice.
You just knew I was going to make a connection to wine reviewing, didn’t you? Well, I am, and here it is: Inexpensive wines are more likely to see spikes in sales from online social media sources, such as blogs and Twitter. Expensive wines are not, because the kind of people who can afford them don’t blog or tweet, and if someone has enough money to buy, say, Shafer Hillside Select ($230 for the just released 2008), they couldn’t care less what some blogger has to say.
However, that well-heeled person considering buying the Shafer does care what the Michael Bauer-equivalent of the wine critic has to say about it. I’m not saying who that equivalent is (wouldn’t be prudent, not opening that can of worms), but I’m reviewing the ‘08 Hillside Select tomorrow, and if I give it a good score, I wouldn’t be surprised if it has an impact on demand.
Steve,
great post, as usual. I have a few comments, if I may:
1. With all due respect, I don’t know who Michael Bauer is (I live on east coast). In general, newspaper reviews are interesting, but only as a point of reference. I would still check what Yelp has to say before I will decide on the restaurant. Also blogger’s reviews are equally important – as long as I have a trust developed with the blog. Social media tools, such as Yelp, give us (all people) more control – at the same time, they also give you more responsibilities. Now I’m responsible for sorting out those “single stars” and understanding what is real and what is not. But when I see 4 stars rating out of 500 reviews, I know I got something very solid.
2. I wonder if that Berkley study took into account demographics and also type and price level of restaurants. What you cited here (outside of Micheline starred establishments) sounds too generic. I would expect that the results will be drastically different based on the particular restaurants, average age of clientele, etc.
3. I’m not in the group which can afford Hillside Select at will – however, it is not unfathomable for me to spend that kind of money on a very occasional treat. When I need to decide on the wine at that price level, ratings matter, but only as a reference. I will actually read what the other bloggers and such have to say about it – I will not make a decision just based on the ratings alone…
Steve:
As you may know, there is a lot of chatter currently (outside the wine world) about the validity of online reviews.
See this recent NY Times article:
http://www.nytimes.com/2012/08/26/business/book-reviewers-for-hire-meet-a-demand-for-online-raves.html?pagewanted=all
And another from about a year ago: http://www.nytimes.com/2011/08/20/technology/finding-fake-reviews-online.html.
By some estimates, more than half of positive reviews on popular review sites may be paid for or outright fake. The issue has attracted the attention of the FTC, as well as academia and private enterprises looking for a work-around.
Just a consideration when basing decisions on online “peer” reviews, even if they are overwhelmingly positive.
Regards,
Brian
Thanks Brian. I’ve covered this issue extensively in the past.
I just can’t use Yelp anymore, not that I used it for anything besides restaurants in area where I didn’t know anyone else to ask. What did it for me was having someone post a complete lie of a review of our store, (of course the guy gave everyone but one place one star, the one place that got five stars? Wait for it, Olive Garden…so there you have it) which isn’t all that crippling considering most people will do as others have mentioned here, look at the other reviews and balance things out but, when I saw a large wine retailer sending out tweets that said, “Who has given us a favorable Yelp review? Come on in and show us and we have a treat waiting for you” which was nothing more than buying positive reviews, all creditability was flushed away. And truthfully, I’m inclined to feel the same way about the majority of bloggers, I don’t get a pap smear at the florist or get my car serviced at dentist…not too likely going to take wine advice from anyone that doesn’t do it for a living or hasn’t been a lifelong student of it anyway. Just my two cents.
Steve
Yelp and Cellar tracker are not Peer Review.
You are misusing this term as you do “double blind” and “single blind”.
Like Sam, I’ve had some reviews posted that were very mean spirited and based on experiences they couldn’t possibly have had (since we don’t make what they’re woofing about, or offer that kind of product). Yelp has an edge to it that can be uncomfortable. On the other hand, when the reviews are positive, they tend to cascade and clear images emerge. We’re finding TripAdvisor is being used by lots of folks visiting wineries, and the reviews seem to be more rooted in reality. Now, about that special offer from my florist: a dozen roses earns a 20% off pap smear….
The beauty of crowd sourced reviews like Yelp is in the number of reviews. If I’m looking for a dinner spot in an unfamiliar city I will put a fair amount of weight to an average ranking when there have been a couple of hundred reviews. If it’s only one or two, I better trust the reviewer, be he Mr Bauer, diningdiva23, or you Steve.
Steve,
Another great topic to ponder. Coincidentally, I was reviewing the hit statistics for my blog the other day and found your premise regarding inexpensive wines to be accurate. On my blog, I have a recurring segment called Supermarket Finds where I review wines commonly available in market chains that are under $15. 90% of the hits I get from Google searches are pointing to these reviews. Less than 1% of the Google search hits point to my reviews of premium wines (premium as in $50 – $90). I don’t bother reviewing wines at or over $100 for the very reason you state: if you can afford those you won’t care what I think. They will care what you think or what Laube thinks but not a blog guy.
Two other interesting trends popped up. One is that searches for mid priced wines are steadily increasing and two a fair amount of searches for supermarket wines comes from the UK. The mid price hits make sense as I know the sales numbers for mids are up but I can’t quite explain the UK data….
Kurt, thanks for trusting me! In my next life I want to be a restaurant critic.
Kurt Burris hits the nail firmly on the head here. Get critical mass in the reviews, and chances are they will be substantially influential.
As for this:
“Inexpensive wines are more likely to see spikes in sales from online social media sources, such as blogs and Twitter. Expensive wines are not, because the kind of people who can afford them don’t blog or tweet, and if someone has enough money to buy, say, Shafer Hillside Select ($230 for the just released 2008), they couldn’t care less what some blogger has to say.”
Couldn’t disagree more. Fallacy of small numbers here, but I can afford that bottle of Shafer, and I would use a friend or blogger’s recommendation to do it, without question, provided I trusted that source. I don’t really trust the sources for the majority of professional wine reviews, because they mostly don’t engage their subscription base at all, and a one-way conversation with an ivory tower just doesn’t engender trust for me. I know you and Paul, for example, from WE, and guess what – yep, trust your opinions on a lot of the wines that you two review, because there’s a combo. of authority and engagement there. I suspect that the same is probably true for a lot of folks who follow my exploits in the on-line critical arena.
Steve, I agree with you about more inexpensive wines getting spikes in traffic from blogs. But what you don’t touch on is part of the reason is that more of those wines appear in blogs because they are inexpensive. The higher the price (and reputation), the less likely it is to be sampled widely. You are one of the few people I know who reviews Shafer Hillside and that is because of WE. I wouldn’t expect to see a wine of this genre consistently show up in a blog unless closely affiliated with an established credible publication.
I used Yelp to check out a car mechanic, and there were some glowing reviews and about an equal number of “worst ever” reviews–but nothing in between. I think it is a matter of motive: people will go out of their way to post when they have a great experience, or when they are really pissed off. But why bother if things were “just OK?’ I suspect the same happens with restaurant and wine experiences.