Fakery of the Crowds

Yaniv Dover shows which online hotel reviews are more likely to be fake, and why.

In reality, however, it’s tough to answer. That’s because “promotional” reviews—those designed not to rate an experience or a product, but slyly burnish your own brand or deface a competitor’s—are not meant to be detected, and distinguishing the true from the false is a difficult task. Yet making that distinction is important, since online reviews have a significant effect on consumers’ purchases.

Computer scientists have created algorithms to identify fraudulent reviews, but with little success. Yaniv Dover, an assistant professor of business administration at Tuck who specializes in social networks and word-of-mouth information diffusion, addressed the question from a different angle.

Dover and research colleagues Dina Mayzlin and Judith Chevalier studied reviews from Expedia.com and TripAdvisor.com, two hotel review sites that have an important difference: anyone can post a review on TripAdvisor, while only consumers who booked at least one night’s stay at a hotel could review it on Expedia. In other words, it’s much easier to write a fake review on TripAdvisor than on Expedia. By examining reviews of the same hotels on both websites, they were able to study the differences in the reviews and measure how hotel characteristics affect the likelihood of review manipulation.

Online reviews have quickly built up a critical mass and gained the power to influence markets, but they are not without problems. For example, in 2004 Amazon.com’s Canadian site experienced a glitch and revealed the true names of anonymous reviewers of books. It turned out that a portion of the glowing book reviews were written by the authors themselves. It’s reasonable to believe the practice has spread to all kinds of products and services. “We wanted to see if online reviews were sustainable, or would they explode at some point and nobody would trust them?” says Dover. When they saw the different platforms of Expedia and TripAdvisor, it looked to them like a natural opportunity for an experiment. “This is something rare that we had to exploit,” he says. “Arranging to study 3,000 hotels would be much harder than just examining these two websites.”

Dover and his colleagues limited their research to reviews of hotels in the 25th to 75th largest U.S. cities, studying 3,082 hotels; 95 percent of the hotels had reviews on both sites. Their data set consisted of 350,430 reviews on TripAdvisor, and 123,569 on Expedia. The hotels were divided into three categories of organizational form: independent, franchised unit of a chain, or company-owned unit of a chain.

These categories are important because they inform the researchers’ hypothesis: that the gains from promotional reviewing are likely to be highest for independent hotels. To test this hypothesis, the researchers looked at the ratio of one-star reviews to five-star reviews and compared the different ratios across the range of hotel types and among the two websites. They also looked at the ratio of one-star and two-star reviews to total reviews for chain and franchise hotels that are close geographic neighbors with independent hotels. In other words, they write, “We measure whether the neighbors of independent hotels fare worse on TripAdvisor than Expedia.”

The empirical evidence was consistent with their claims: Dover and his colleagues found that independent hotels engage in more review manipulation (positive and negative) than franchised and company-owned hotels. Although that might sound damning, the authors advise that “while the total amount of review manipulation is economically significant, it is still small relative to the total amount of reviewing activity.”

The lesson for consumers is clear: the hotel reviews you use to determine where to stay are probably real, but it pays to compare reviews in multiple sites to check for consistency. “What we suggest is, if you do have validated and non-validated sites,” says Dover, “you can always use the validated site as a checkpoint.”