The Billboard Effect is dead, says a study of hotels listed on OTAs

A new study on how US travelers book hotels online contradicts earlier research on the topic. It suggests that a much-touted “billboard effect” for hotels listed on online travel agency (OTA) websites may no longer exist.

The study of about 50,000 online travel shoppers in 2014 found that users who browsed hotel sites tended to ultimately purchase their hotel stays via intermediaries, such as Expedia,, and TripAdvisor, on average.

The behavior was not true in reverse. Internet users who began their browsing on OTAs and similar intermediary sites tended to purchase hotel stays through those intermediaries, too.

The study was teased during a panel talk at the Revenue Strategy Summit held on Wednesday in Washington, D.C.

Different study, different sponsors, different results

The just-released study was sponsored by the American Hotel & Lodging Association’s Consumer Innovation Forum, many of whose members are leery of the large commissions they pay to intermediaries.

The data was analyzed by P. K. Kannan, chair of the department of marketing at the Robert H. Smith School of Business at the University of Maryland.

The results were presented at a conference hosted by the consultancy Kalibri Labs, the revenue management software firm Duetto Research, and marketing-and-event agency Silver Hospitality Group.

Changed consumer behavior?

The study’s results suggest a shift in consumer behavior since 2009, when Chris Anderson, an associate professor at Cornell University, published a study about Expedia and hotels in the JHM Hotels.

His study found that the appearance of a hotel chain on the first page of results on Expedia coincided with a rise in reservations through JMH Hotels’ own websites. The direct booking uplifted ranged between 7.5% and 26% uplift.

His follow-up April 2011 study of 1,720 reservations at InterContinental Hotel brands found a similar behavior. For every commissionable hotel reservation that came through Expedia, there were between three and nine reservations at an IHG-related website that were influenced by the hotel having been listed on Expedia, on average.

The studies were touted by Expedia and others as proof of a so-called “billboard effect”, where OTAs pull double duty as a kind of search engine marketing tool for hotels’ own websites.

Bye-bye, billboard effect?

Unlike the Cornell work, which used data from Expedia and hotel chains directly, this study used clickstream data from a market research firm and it was analyzed via attribution modeling.

Yet like the Cornell research, this study did not include mobile device users.

Kannan declined to share details of the study with Tnooz prior to the report’s publication this winter.

What follows is a transcript of what Kannan told the attendees of the conference. Note: When he says “hotel sites” he means “well known US hotel chain sites”, such as or”

“Here’s a quick snapshot of what we found in a preliminary analysis of data.

We are looking at how people are searching online in their search and booking process. What we have is data about their clickstream behavior from a syndicated market research firm.

We looked at two different years, 2012 and 2014, to see how behavior is changing.

We did a path analysis. We track how long users visit sites, how many pages they visit at each site, and whether they complete a transaction via that site, such as an intermediary like Expedia. If they booked a hotel, we know what brand they booked.

The market research firm gives us session data as well as the transaction data.

Once we know there was a conversion, we merge that particular point with the transaction data, which gives us a lot more information about what they did, such as which brand hotel they booked a stay with and if they also booked a car or a flight.

We have done an individual-level analysis of visitors to find out how loyal they are to various domains.

We track each individual user and see whether, if they visit a hotel site, what are the chances they’ll book at the hotel site, and if they visit a hotel site, what are the chances they’d book at an intermediary site.

We’re able to map how loyal consumers are to particular domains. We’ve created three broad categories: Those who exclusively visit hotel sites, those who exclusively visit intermediary sites, and those who visit both domains.

Between 2012 and 2014, the number of people who exclusively visit hotel sites has come down from 12% to 10%. Whereas people who visit exclusively intermediary sites have gone up from 48% to 60%. People who go to both types of sites have dropped from 40% to 30%.

We then drilled down on the data about people who visit both domains and checked to see what kind of behavior they exhibited.

We tried to figure out how many booked at hotel chains versus those who booked at intermediary sites. What we find is that, between 2012 and 2014, the number has increased for hotel chains from 7.9% to 10.8%. The number for intermediaries has dropped from 7.7% to 6%.

There is not an appreciable change in the percentage of people who browse at both types of domains that then book at both types of domains.

This is not good news for hotels, despite first impressions.

Note that we found that the segment of shoppers who go to both types of sites has dropped from 40% to 30% in the past two years.

People who in 2012 were willing to visit both domains are in 2014 opting disproportionately to use intermediaries exclusively.

We also looked for search behavior. How many visits happen at intermediary sites and how many visits happen at hotel sites? 82% end up at an intermediary shop for browsing, while hotel sites get 18%.

Here is another look at the data. What happens between two consecutive visits? If you went to a hotel site, where do you go later? Do you go back or go somewhere else?

Some people repeat visit hotel sites. That demographic has increased from 57.4% to 62.9% over two years.

The people who leave hotel sites and go to intermediary sites later was 42.6% and dropped 37.1% (the complements of the previous numbers).

At first that seems like good news for hotels.

But the repeat visits for intermediary sites is fairly high: 90%. If you go to an intermediary site, there’s only a 10% chance your next visit will be a hotel site.

We tried to look at each individual’s string of path data and relate it against it against conversions.

The metric we used for this is called Generalized Transfer Entropy (GTE). What this technique does is let you, at the individual level and in a statistically significant way, indicate whether a user is more likely to go to point B if he or she went to point A.

In 2012, users who browsed hotel sites had a significant directional tendency of transacting at those hotel chain sites.

In 2014, this changed. Users who went to a hotel site in 2014 were more likely to actually convert at intermediary sites.

Meanwhile, intermediaries retained their popularity.

In 2012, users who went to intermediaries tended to book at intermediaries and were very unlikely to book at a hotel site instead. This held true in 2014….

To say that differently, in 2012, people who browse to hotel sites have a significant directional impact to actually transact at the hotel sites.

If you went to an intermediary site, you were very unlikely to go to hotel site to transact. It’s a negative impact on you transacting elsewhere. If you book some other travel product at an intermediary, such as a rental car, it has a negative impact on you booking a hotel at a hotel site.

In 2014, if you went to a hotel site, you’re also more likely to go book at an intermediary.

We have a lot more data to go through, such as at the chain-level and intermediary-by-intermediary.”

The data is expected to be released sometime between late 2015 and February 2016, said Cindy Estis Green, CEO at hospitality analytics consultancy Kalibri Labs and a co-organizer of the conference. Results from the study will be incorporated into the consultancy’s annual Distribution Channel Analysis report.

THE CORNELL REPORT: Does the Expedia billboard effect still exist for hotels?

A YEAR AGO: Study: At upscale hotels, guest acquisition costs are devouring room revenue growth

NB: Image courtesy seanoneill/flickr

Share on FacebookTweet about this on TwitterShare on LinkedInEmail to someone
Sean O'Neill

About the Writer :: Sean O'Neill

Sean O’Neill had roles as a reporter and editor-in-chief at Tnooz between July 2012 and January 2017.



Leave a Reply to Is the Billboard Effect Alive, Dead or Just in Limbo? Cancel Reply

Your email address will not be published. Required fields are marked *

  1. Business Model Canvas

    […] themselves. This effect is called “billboard effect.” More recently, other studies have questioned that this effect (still) exists. It is ultimately hard to measure due to the hundreds of micro moments over which […]

  2. Come sfruttare per ricevere gratuitamente prenotazioni: il BillBoard Effect. Ma è vivo? - BookingLight

    […] metterci questo tarlo nella testa è stato uno studio recente condotto da P.K. Kannan della  Robert H. Smith School of Business dell’università del Maryland […]

  3. Is the Billboard Effect Alive, Dead or Just in Limbo?

    […] then, Tnooz reported on a big study that claimed the billboard effect was a myth. That study, which is slated to published later this year, followed 50,000 online travel shoppers. […]

  4. Forthcoming Study: Is The Billboard Effect Dead? - TravelTech

    […] The Billboard Effect is Dead, Says a Study of Hotels Listed on OTAs July, 30 2015 Sean O’Neill TNooz […]

  5. Bob

    These findings seem to fly in the face of common sense. If I am looking for a hotel in a city I have never been to, where do I start? Do I go through all of the organic search results in the city, then randomly pick one that ‘sounds good’ without knowing anything about pricing or availability? How can I see the actual situation in the complete market, before making my final educated choice? Many consumers are conditioned to check the hotel website as part of the process, although there is usually not much of an advantage in doing it. So there must be at least a portion of people who just end up booking direct. I know, because I have done it myself. How can it be then that there is no impact? What are hotels smoking that makes them think that all their business is just going to go to them directly if the ‘evil’ OTAs go away? It suggests that the OTAs add no value. The valuation of Priceline would suggest otherwise…

  6. Hedwig Wassing

    Sean, thanks for adding antother piece to this complicated puzzle, where many parties have conflicting interest while being partners at the same time.
    Now the rate parity issue is tackeled (more so in France than in other parts), there is a great opprtunity for hotels to regain control. As I have indicated in earlier responses to articles covering these issues, hoteliers now should be working hard on matching or surpassing the friendly user experience of booking a room at and other sophiticated OTA’s. Investing in that would be my #1 prority now.
    As hotels have control over the service delivery they are able to fullfil exciting promises that OTA’s can never guarantee.

  7. Margherita

    The thing is that many independent hotels, unless they show up on, Expedia or TripAdvisor, simply do not exist… they do not have the resources and the budget to be found in today online scenario.

    Organic results have been pumped up with google stuff and there is no other way to get visibility… for me the only way to understand what really is going on would be to take a large amount of hotels in different markets (Europe is different from US, for example) that are already listed on OTAs and check what happens if they do not show up there for a month, maybe in different periods of the year.
    The results in many cases that I have seen is a sharp decrease of the bookings on the official website… how would they call this?!

  8. William Beckler

    The older study is a controlled experiment, while this study is a data mining exercise. The problem with clickstream analysis is that correlation is confused with causation, while a controlled experiment is the only way ever to determine causation.

  9. Dave Spector

    Great reporting as usual Sean.
    A note of caution on this: according to the article, the Study in question only compared behavior on large flag/chain portals, which offer little differentiation from vanilla OTAs. I’m sure the behavior would be markedly different for boutiques and independents where consumers want the peace of mind of a direct booking.

    Dave Spector
    T A M B O U R I N E
    New York City • Fort Lauderdale

    • revguroo

      I completely agree. The result is not really surprising when looking only at brand sites, many of which are very far from a modern user experience. The picture may be very different for independent hotels with limited marketing budgets and resources. Yet, resourceful revenue managers might just be able to win the OTA customer back…

  10. really

    I am sorry to say that making a link between the two studies is just a sensationalist effort to attract attention. It’s just a shame that in the hotel world many people choose to go the populist propaganda route to stir up emotions amongst hoteliers. These are two completely different studies, with different methodologies, measuring different things. From what I read in this article the studies may complement each other indirectly to understand online hotel stay purchasing behavior but they do not contradict each other.

    If you want to know if the billboard effect still holds true or not then do the exact same study again. Do not come up with a completely new study and say it contradicts the other study. Tnooz I love that you tell me all that’s going on in the travel tech world but I do not like your subjective reporting. I have seen it again and again, tnooz has a certain bias and objectivity from it’s authors would be welcomed. Sadly this comparison comes to mind, Tnooz is the fox news of the hotel news world.

    • William Beckler

      That doesn’t seem like a fair critique to me. Tnooz is doing a great job of presenting both studies, and it links to both studies. It is not endorsing but rather extensively quoting the new study. And like every media site in the world, it is trying to fan the flames of debate by being a little extra controversial in its framing, but that’s normal, and Fox has no monopoly on that.

    • Sean O'Neill

      Sean O'Neill

      Sorry to disappoint but I’m afraid I plan more of the same. We showed a report from one side, and a year or so later, we’ve showed a report from the other side. That’s an effort at balance, not bias.

      After yesterday’s presentation, I overheard multiple people in the crowd around the buffet table saying “That’s the end of the billboard effect.” I’m not the only person to have that intepretation.

      Yet I completely agree: To repeat the earlier study would be the most relevant thing to do. If Cornell does, we’ll happily report it.

      But this new study seems to capture other interesting data, too.

  11. Mark Scott

    All this research goes to prove that most travelers are not booking hotels in the purchase path ie they are booking hotels after they have purchased airline tickets. This coincides with all the studies we have carried out.

    Getting travellers to go to the desired site after they have purchased air travel is naturally placed after the air purchase – unless they are buying a hotel/flight/rental car bundle. Even then those who don’t book a hotel can be made a suitable offer when they are in the mindspace to book.

  12. Roger Smith

    Where is the source of this clickstream data? What elements are they tracking? We monitor the analytics of over 10,000 hotels and I can assure you this far from accurate. These properties still sees large % bumps of traffic for priority placement on the OTAs.

    • Sean O'Neill

      Sean O'Neill

      Data will be released this winter. We were not able to get access to it at this time.


Newsletter Subscription

Please subscribe now to Tnooz’s FREE daily newsletter.

This lively package of news and information from Tnooz’s web site provides a convenient digest of what’s happening in technology that drives the global travel, tourism and hospitality market.

  • Cancel