We like to run lots of experiments in and out of campaigns to test how users behave in metasearch and understand how we can help drive our clients more bookings at higher ROIs. In a recent experiment we set up a controlled metasearch environment and tracked how users responded to price, position, and advertiser-types.

You can get a sense of this experience through the mockup below.

By running this experiment we wanted to answer a few interesting questions:

• What is the impact of small and large price differences on ad CTR?
• Which users choose Suppliers vs. OTAs, and why?
• How much does position affect CTR in parity and non-parity scenarios?

Price has the highest impact on selection and CTR

It’s not a great surprise that in a value-conscious marketplace, users most frequently choose the offer with the lowest price. As you can see in the data below, users also are more likely to choose the lowest price when it’s a better deal.

Another way to think about this is that according to this experiment the “low price” in a metasearch ad auction is 8x more likely to get clicked.

Ad position’s impact on selection and CTR

If you’ve spent much time in the digital marketing world you know that typically CTR declines significantly with ad position, and metasearch is no different.

Our experiment allowed for an ad with a higher price to be shown in position one (the Left position) in order to measure whether position can draw users away from a lower price offer.

We believe this has been shown in the data above; if position had no impact on selection and CTR, we would expect 9% of users to have selected the left offer in the “Lower Price Right” scenario instead of 15% of users as was observed.

Users prefer Suppliers to OTAs

We’ve seen in previous experiments that there is some user preference towards Suppliers (hotel brand direct) vs. OTAs, but haven’t been able to measure it this as cleanly in the past.

When everything is the same, we observed a meaningful bias to Suppliers. This continues to exist in scenarios where the Supplier is more expensive. A 1% difference in the rate only had a small impact on the selection rate of Suppliers. Only when there was a 10% difference in price did the selection rate begin to even out.

Preference goes both ways, and other important learnings

We were lucky to be able to survey users after their selection and found that brand preference worked for OTAs as well, just not as frequently as it did for Suppliers. Here is what users self reported as the reason for their booking choice:

As you might expect by now, Price was confirmed again as the greatest factor in ad selection. Trust and Supplier Preference were second and third. Trust is interesting to us because it often benefits the OTAs; users were very familiar with these brands. While overall we saw a preference for Suppliers, there were definitely users that cited much greater trust levels for the OTAs.

Themes that surfaced in areas of Supplier preference included better treatment, better customer service, price matching, and the assumption that booking direct would lead to more upgrades. Users that showed Supplier Preference would often mention bad experiences with OTAs, and assumed that booking with an OTA would be more expensive in the long run, result in worse service, get them into a lesser quality room, or cause other issues with their booking.

In OTA preference, users cited better selection, deals, and better rooms for the prices. They often would call out that the OTA they chose was a trusted brand and in many cases cited recent TV ads as a source for that trust. Some users felt that they would get a better room as a result of booking with an OTA.

Making the data actionable

This was a fun experiment to work on because it nicely confirmed some assumptions that we frequently hear from the general industry and exposed some new insights for us.

• You can’t overstate the impact of price on ad selection and click-through-rate in metasearch. For OTAs, this means making sure that the best rates are available throughout the inventory and adds some context to the value of enforcing price parity agreements. For Suppliers – especially chains – this means making sure that your book direct price is not undercut through any other providers and channels. Advertisers tracking and enforcing parity are going to fare much better in the long run.
• Ad position – especially on engines that don’t just expose the lowest rates – is a very important marketing metric to pay attention to. We often tell clients an easy rule of thumb for click curve is 50% per position. Understanding positioning on top of pricing scenarios adds key context to results.
• There are still important knowledge gaps with users that are important for both OTAs and Suppliers to cover through marketing and education. These “in the middle” users are up for grabs, especially in cases of parity or near parity. Through better messaging and offer communication, either advertiser group could certainly capture more share.

If you have any questions on this experiment or want to see the data split in a different way, please comment below or e-mail us directly.