Test #4: “Adding a caution message”

We mentioned in a previous post that users typically have one goal when they come to a site, and purchasing ancillaries is usually not that goal. The challenge becomes how to get the customer’s attention and provide a compelling reason for them to consider adding an ancillary. Most of us are accustomed to seeing system feedback messaging (errors, warnings and important info) that utilizes eye catching colors and symbols to draw our attention and inform us. Coupling this emphasis technique with a message that is short and to the point can be an effective way to get users to consider your offer.

Goal:

Provide an initial state message when a customer is presented with an insurance offer that informs them their trip is not protected. This messaging will cause the customer to evaluate the need and consider adding the product.

Key Performance Indicator (KPI):

Conversion Rate

Traffic Source / Segment:

Desktop traffic only, non-US, airfare/hotel travel package purchaser segment

Summary of difference(s) between variations:

Figure 1: Variation A – Original  (48% of traffic)
Figure 2 – Variation B (Test) (47% of traffic)
*** WINNER ***

 

Sample Size & Test length:

Approx. 208,760 visitors were randomly shown either variation over the same 18-day period in 2017.

Hypothesis:

By introducing a caution message with the initial offer we will increase purchases and therefore revenue. Providing a caution message draws attention to the offer because it utilizes colors and icons commonly associated with important information (leading exclamation point and highlighted background). The message content encourages the customer to recognize that their trip is not protected with insurance, which they might not have considered otherwise.

Results:

The test variation B increased revenue by € 60,984 annualized which is +30.41% with 97% significance.

Actionable Takeaways:

The test results supported the original hypothesis: Customers positively responded to the message and significantly lifted conversion rate and revenue.  Few customers take the time to read all the content on a web page and typically will scan a page to focus on what they consider their primary objective. Utilizing commonly used colors and symbols associated with system feedback (like warning symbols and yellow/red backgrounds or font colors) can be an effective way of drawing attention to your content. Providing a simple message that can then ‘plant a seed’ may be just enough to get additional customers to consider your offer.

see other tests