How taking a softer call to action affects donations
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 12/26/2014 - 12/27/2014
As it was getting closer to the end of the year, Heritage wanted to test a messaging approach with their fifth email in their year end campaign. Toward the end of urgency-based marketing campaigns, it is common practice to make a direct ask in the email. The question was how early can that direct ask be made and have it still be effective.
Research Question
A softer call-to-action within the email will drive more traffic to the website which will, in turn, convert a greater number of donors.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Direct Ask | 0.03% | ||
| T1: | Soft Call-to-Action | 0.03% | -4.3% | 11.6% |
This experiment has a required sample size of 15,418,569 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The softer language in the email increased the number of visitors by 41% but it actually converted fewer total donors in the test. At this stage in the campaign, there does not seem to be any impact to response rate by utilizing a softer call to action.