Consumers often can't detect fake reviews – and underestimate how many negative reviews might be fakes

The Research Brief^[1] is a short take about interesting academic work.

The big idea

Consumers who have a personality that scores high in terms of openness – such as being open to new adventures and intellectually curious – have better success at spotting fake reviews than other personality types, according to our recently published research^[2]. Extroverted people, on the other hand, tend to have a harder time identifying a fake review.

To reach these conclusions, we compiled reviews from a unique data set of 1,600 Chicago hotel reviews, marked as either fake or real, that was compiled by artificial intelligence engineer and researcher Myle Ott^[3] and his team for peer-reviewed research they published in 2011^[4] and 2013^[5].

Ott and colleagues mined real reviews from travel review websites such as Tripadvisor, Hotels.com and Expedia, which have a reasonably small deception rate. They gathered fake reviews by using Amazon Mechanical Turk to recruit people to write fake hotel reviews that sounded truthful.

We then used Amazon Mechanical Turk^[6] to recruit 400 participants and asked them to imagine they needed to choose a hotel for a planned trip to Chicago. Each participant was assigned a hotel, read eight reviews about it, guessed at which ones were fake and explained why they seemed fake or real. The eight reviews were a balanced set of two positive fake, two positive real, two negative fake and two negative real, shown in a randomized order.

Participants then answered questions that allowed us to assess where they rank in terms of the big five personality types^[7]: extroversion, agreeableness, openness, conscientiousness and neuroticism.

Overall, we found that consumers generally trust negative reviews more than positive ones. In evaluating whether an online review is genuine, consumers frequently underestimate the number of negative reviews that can be fake, while assuming that some positive reviews might be fake.

When we asked our participants why they thought a negative review was trustworthy, we found that they didn’t fully take into account that the writer might be motivated to post the review out of a desire to harm the business – for example, hostile competitors or angry customers.

We also found that readability, length and content affected perceptions of the review. Study participants were more likely to trust positive reviews when the sentences were short, and more likely to trust negative reviews when the sentences were long.

Shorter negative reviews with less emotional content were also more believable.

And in terms of personality type, while participants scoring high on openness were best at spotting fake reviews, and those with more extroversion did the worst, it was only for positive reviews. All personality types did pretty poorly at weeding out fake negative reviews.

Why it matters

Consumers consider online reviews to be among the most important sources of information^[8] for making buying decisions. However, according to a 2019 report by Which?^[9], the U.K.‘s consumer champion organization, many of these reviews on popular websites such as Amazon are fakes.

But consumers are consistently bad at detecting^[10] fake reviews in their buying decisions.

Our research could help consumers become more aware of how they respond to reviews, especially negative ones.

What still isn’t known

Our research has identified certain features in the length, wording and structure of online reviews, as well as consumer personality types, that lead consumers to trust online reviews. We still don’t know why these features convey trustworthiness to consumers or why they differ for positive versus negative reviews.

References

1. ^{^} Research Brief (theconversation.com)
2. ^{^} our recently published research (doi.org)
3. ^{^} Myle Ott (myleott.com)
4. ^{^} published in 2011 (doi.org)
5. ^{^} 2013 (aclanthology.org)
6. ^{^} Amazon Mechanical Turk (www.pewresearch.org)
7. ^{^} big five personality types (www.thomas.co)
8. ^{^} most important sources of information (doi.org)
9. ^{^} a 2019 report by Which? (www.which.co.uk)
10. ^{^} are consistently bad at detecting (journals.sagepub.com)