List of all editorials

E-commerce - sell more with irrelevant purchase options?

By Richard Whitehand, december 2013

A few years ago The Economist published an article refering to an experiment that the behavourial economist Dan Ariely conducted in relation to the Economist.com website. The Economist at that time offered 3 subscription options on their website:

The Economist - subscription alternatives

A. Economist.com subscription - $59
One-year subscription to Economist.com
Includes online access to all articles from The Economist since 1997.

B. Print subscription - $125
One-year subscription to the print edition of The Economist.

C. Print & web subscription - $125
One-year subscription to the print edition of The Economist and online access to all articles from The Economist since 1997.

<

The fact that the "Print subscription" and "Print & web subscription" were the same price was puzzling and Dan Ariely conducted a survey of 100 management students to see which option they preferred. The results were as follows:

A. 16%

B. None

C. 84%


Dan Ariely then repeated the study but with the unpopular option removed (i.e. option 'A') and the results were surprisingly different:

A. 68%

C. 32%


In other words, the presence of option 'B' in the first case had apparently radically affected peoples' decision making processs by making option 'C' (the combined print and web subscription) look like a better deal. For those interested, further information about this can be found in The Economist's article and by watching Dan Ariely asks: Are we in control of our own decisions? (TED talk).

Of course this finding is very interesting - by simply including the irrelevant option 'B' then one can conclude that The Economist would make more money on their subscriptions. Indeed, since then, this finding has been picked up on and referred to in a variety of other articles and presentations.

We therefore decided to construct a similar test ourselves. We surveyed a broad range of Internet users (approximately 300 in total) to choose a purchase option for a film they wanted to buy.


We surveyed half the participants first, who were given the following format options for a film they wanted to buy:

A. File download (electronic copy only) - 89 SEK

B. DVD only - 179 SEK

C. DVD and downloadable file - 179 SEK


Of the 133 participants who completed the survey, choices were as follows:

A. 59%

B. 9%

C. 32%

We anticipated that option B ('DVD only') would be very unpopular, though infact 9% of people selected it. Upon examining the data we found that nobody under 30 years old had selected 'B' - a similar result as with The Economist subscriptions, as those surveyed were students. (One must assume that some older users don't care whether they get a downloadable file or not as they perceive it as irrelevant).

Our next step, as with the study conducted on The Economist subscriptions, was to repeat the survey with option 'B' removed.


Of the 155 participants who completed the survey with only 2 options, choices were as follows:

A. 60%

C. 40%

Surprisingly, the proportion answering option 'A' remained unchanged.


Now, there may be a variety of reasons as to why the results are different in this case - it would be very interesting to explore them further. Part of the reason may relate to the different nature of the question and answer options. However, regardless of this, these findings highlight two very important points:

  • User behaviour in one situation doesn't necessarily translate to the same behaviour in another situation, no matter how similar things may look on the surface. It is critical to test different alternatives.
  • A/B testing lots of different option combinations won't help you unless you understand the behaviour and interaction that is going on. Even if you do and eventually stumble upon options that provide a better outcome, how do you know this is as good as it can be? How do you work out how to improve them further?

Related articles:

Did you find this editorial interesting?

You might like to read some of our other editorials.
Please send us an email if you have any comments or suggestions!