If restaurateurs didn't already worry about Yelp, they're about to. A new study published in the Economic Journal claims that as little as a half-star difference in a restaurant's average rating on Yelp greatly impacts its reservations each night.
The study, which examined San Francisco area restaurants, used empirical data to back up the widespread belief that internet review forums influence consumers to an incredible degree:
We implement a regression discontinuity design to estimate the effect of positive Yelp.com ratings on restaurant reservation availability. An extra half-star rating causes restaurants to sell out 19 percentage points (49%) more frequently, with larger impacts when alternate information is more scarce. These returns suggest that restaurateurs face incentives to leave fake reviews but a rich set of robustness checks confirm that restaurants do not manipulate ratings in a confounding, discontinuous manner.
The study contradicts earlier findings from a retail research firm that said that only 14 percent of a diners' motivation could be linked to online reviews and that fine dining restaurants suffered minimally from bad Yelp reviews.
UPDATE: Jeremy Magruder, one of the study's authors, provided the following description of how the study was performed. It explains what led the researchers to determine that Yelp ratings had a causal effect -- and not just a correlative one -- on restaurant reservations:
People who submit reviews to Yelp submit something which is 1, 2, 3, 4, or 5 stars. Yelp takes these together and aggregates them to create an average rating for each restaurant. Then, it rounds that average rating to the half-star level. So restaurants are displayed with 3, 3.5, or 4 stars.
The true average rating, of course, is continuous and could be any number between 1 and 5 - so for example there are some restaurants have true ratings of 3.74, while others have a true rating of 3.76. We might think these restaurants are pretty similar in terms of food quality, service, etc. - whatever people take into consideration when making a review, they've ultimately reviewed these two restaurants almost exactly the same. But because Yelp rounds its ratings, the 3.74 restaurant will be displayed as 3.5 stars, while the 3.76 restaurant will be displayed as 4 stars. By making comparisons like that, we can see where there are differences in reservation availability between two restaurants which are almost exactly the same in terms of how customers review them, but which look very different on Yelp. It's these types of comparisons which let us say that the only "real" difference between these two restaurants is Yelp - and when we compare restaurants like these, we see that the restaurants which just barely get 4 stars sell out about 19% more frequently than restaurants which almost get 4 stars. This is what lets us infer causality.
Correction: An earlier version of this article claimed that the study was unable to demonstrate causality based on a quote from the researchers taken out of context in the Guardian. The post has been updated with a statement from one of the study's authors describing the process and how it allowed them to infer a causal relationship.
Related
Before You Go
