Time of Grace

How offering a book as a premium with a donation vs at retail price impacts conversion rate and overall revenue

Time of Grace

Experiment Summary

Ended On: 02/16/2024

For Time of Grace’s monthly book promotion, we hypothesized that offering the book in exchange for a donation would result in increased transactions and revenue versus just offering it for purchase. Historically, Time of Grace has promoted their new books on their Shopify store at retail price. This test was accomplished by making a change to two areas of the funnel. The control vs treatment emails would have a change in the call-to-action copy. The control (for purchase) would lead them to Time of Grace’s Shopify page, whereas the treatment would lead them to a donation page that had the book as a premium on it.

Research Question

We believe that offering a new book as a premium gift in exchange for a donation rather than as a product for purchase for Time of Grace’s donor file will achieve an increase in transactions and revenue because users are more likely to be motivated by the generosity of the organization, and the value of receiving the gift for a donation.. because users are more likely to be motivated by the generosity of the organization, and the value of receiving the gift for a donation..

Design

C: Control

T1: Treatment #1

Results

	Treatment Name	Open Rate	Relative Difference	Confidence	Average Gift
C:	Control	0.32%			$36.89
T1:	Treatment #1	0.61%	92.2%	99.5%	$57.40

This experiment has a required sample size of 4,151 in order to be valid. Since the experiment had a total sample size of 17,057, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

92.2% increase in traffic
× 0% increase in conversion rate
× 55.6% increase in average gift
199.0% increase in revenue

Key Learnings

The key learning from this experiment is that offering a new book as a premium gift in exchange for a donation, instead of selling it at retail price, significantly increased the number of donations received. The treatment group saw a 92.2% increase in donations compared to the control group, with a high statistical level of confidence.

Not only did this result in an increase in donor/purchaser conversion rate, it also resulted in a valid 199% increase in revenue with a 98.9% level of confidence.

Control: 27 Purchases | $996 in Revenue | $36.89 Average Gift

Treatment 52 Donations |$2,985 in Revenue | $57.40 Average Gift

This approach also did not limit the ministry’s impact in actual copies “sold” with 53 units being purchased under the control and 55 units being requested as premiums with a donation.

This indicates that users are more motivated by the generosity of the organization and the perceived value of receiving a gift in exchange for their donation. Therefore, in future experimentation, it would be beneficial to continue offering premium gifts as incentives for donations.

This could be applied to other products or promotions as well, where offering a premium gift or exclusive content in exchange for a donation or purchase may motivate users to take action and increase overall transactions and revenue. The organization should consider implementing this strategy more frequently to leverage the increased motivation and generosity of their audience.

Experiment Documented by Tristan Amaya

Question about experiment #58424

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.