How the kind of offer in an appeal affects donor conversion.

Harvest Ministries

Experiment Summary

Timeframe: 08/15/2016 - 08/22/2016

Normally, for Harvest’s mid-month appeals, they offer a resource (normally a book or DVD) to people if they give a gift to the ministry.

For Harvest’s August mid-month appeal they ran a test to see if they could acquire more donations through offering a book (the Tell Someone book) or through a free online course (the Tell Someone course). For the book offer, people were taken directly to a donation page where they could get the book and give their donation. For the course offer, people had to first sign up for the course and then be asked for a donation. Using 40% of the email file, they A/B split tested this.

Research Question

Would the Tell Someone course offer receive more donations than the Tell Someone book offer?

Results

This experiment has a required sample size of 68,181 in order to be valid. Since the experiment had a total sample size of 85,031, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

While the results didn’t validate, the initial data pointed to validity and the treatment (the Tell Someone course) had an increase in opens, clicks, donations (25%) and revenue. Using this data, they then sent the remaining 60% of the email file the Tell Someone course offer.

Even though the test didn’t validate, deciding to send the remaining file the Tell Someone course offer was not only the best decision from what we were seeing with initial donations and revenue, but the long-term benefit and value of these donors is so much greater with the course than it would be with a more transactional offer like the book. The course gives people the opportunity to engage with Harvest on an ongoing basis at a deeper level. And at the same time, it allowed them to instantly support the ministry. By offering people something out of the ordinary that had a higher perceived value, people were much more motivated to give through the course offer than the book.