Researchers at a US university have developed software which they claim can identify fake hotel reviews by analysing its linguistic structure.
The group from Cornell University in New York state claim its technology tested 800 reviews of hotels in Chicago and was able to pick out “deceptive reviews” with close to 90% accuracy.
This was achieved by asking a group of people to deliberately write fake positive reviews of properties which were later compared with an equal number of verified truthful reviews.
The university says human beings are “lousy” at identifying false reviews online (it asked a judging panel to try), but the software analyses sentiment, word structure and grammar within a review to establish whether it should be flagged up.
“Truthful hotel reviews, for example, are more likely to use concrete words relating to the hotel, like ‘bathroom’, ‘check-in’ or ‘price’.
“Deceivers write more about things that set the scene, like ‘vacation’, ‘business trip’ or ‘my husband’. Truth-tellers and deceivers also differ in the use of keywords referring to human behavior and personal life, and sometimes in features like the amount of punctuation or frequency of large words.
“In parallel with previous analysis of imaginative versus informative writing, deceivers use more verbs and truth-tellers use more nouns.”
One of the students involved in the project, Myle Ott, cautions that the work so far is only validated for hotel reviews (presumably because the technology could be applied to other verticals where reviews are used) and on the subset of properties in Chicago.
Although companies such as TripAdvisor claim to have clever software behind the scenes to help identify fake reviews on their sites, Cornell students see the technology being applied in the industry perhaps as a “first round filter”, allowing sites to score potentially false reviews which require further investigation.
Related posts:
About Kevin May
Interesting, I often find myself wondering which tripadvisor reviews are fakes; any mention of “all rooms have xxx” makes me suspicious (which guest visits all rooms?) as well as the overly glowing/damning ones. I often look up how many other hotels that person has reviewed, and with what scores. As far as I know, the only checks tripadvisor performs are to query reviews posted from a similar geographical location as the property, which stops a hotel praising itself (or damning its neighbours) but does not stop them asking non-local friends to do both on their behalf. At i-escape.com, we only post reviews from people who have booked and paid through us, so we are sure they are genuine.
As a matter of interest, how did the students know which of their 800 reviews were indeed fakes?
My understanding of the test, Michael, is that they asked 400 people to write fake reviews for a selection Chicago hotels, and then compared these to 400 genuine reviews – this still leaves the question of how they verified the “real” ones though!
I like the concept of only allowing reviews to be made following a booking – maybe this can be broadened into a type of “token” system, say you have a unique code on your bill which can be entered into TripAdvisor to make a review “verified stay” – may still need controls to prevent self promotion but at least would diminish the chance of competitor negativity.
I like the token idea – but does tripadvisor have enough clout to persuade every hotel on its books to issue a unique code for each confirmed booking (and presumably then send them all to tripadvisor so they can be cross checked when new posts are submitted)? Or do you mean a s.th like a password which only changes, say, every month? Anyway, food for thought.
I agree enforcing such a system would take considerable clout (I clearly spend too long online as I thought you misspelled Klout…) I wouldn’t expect such a system to be universally required, but a way to add value and authenticity to reviews.
It might not even need to be too complex or require integration with billing – when you check out the staff can plug in your name and dates of stay into a TripAdvisor site or app, this generates the unique code which can even be handwritten on the bill, the guest can then redeem this for a verified review at their convenience.
TripAdvisor could motivate establishments and guests to use the system by adding some incentives, maybe giving the reviews a special icon or weighting of some kind. These would demonstrate that the hotel is encouraging reviews (so hopefully feel they have nothing to hide) and that the reviewer has actually stayed there.
It could even be sold as a timesaver for guests, enter your code into the TA website and it has all the details of your stay pre-filled for you!
Great post. One of the best ideas I have heard of in a very long time. Are they going to build a business out of it?
I hope the Cornell kids check for grammar. Most phoney reviews I have read are crappy at grammar and punctuation.
Not true at all. Our website only posts reviews from actual guests who are invited to post reviews and the spelling, grammar, and punctuation is generally very poor.
The one problem I can foresee with such software or algorithms is that in the same manner that antivirus software companies constantly try to catch up with hackers, so will software such as this require continuous updating to catch up with fake reviewers. They learn and adapt very fast.
Most hotels have the ability to question a posting if they think it is fake with Tripadviosr. However TA does not do a validity check with the customer that post that post the review to verify if they can provide proof of stay…perfect example look at any hotel that is in the Pre-Opening phase ( taking reservations prior to the opening) but have a Tripadvisor listing and you will see how many reviews are posted where people actually lie about their stay and meanwhile the hotel is undergoing construction.
Check out NYC, Los Angeles or any major city with hotels listed on TA and you will see. Therefore this study questions the validity of some of the Tripadvisor postings..
I have never gotten a query from ta.
Good initiative in the right direction. It´s pretty clear that such a tool is much more than expected in the short/medium term.
We had a completely phony review posted on ta and they did nothing about it. I applaud anyone who even thinks about how to protect innocent buninesses from false reviews. Unfortunately, there are some pretty hostile people out there who know how to attack someone’s reputation and are unable to be held accountable for it. No one likes to see a negative review, but at least we can learn from the honest ones and we also learn that it’s impossible to please everyone, all the time.
One of he best idea I’ve heard of in my life. It’s a great concept if you think about it.