Dallas Theological Seminary

How using a ‘donation interruptor’ affects recurring giving rates during a Giving Tuesday high-urgency campaign

Experiment ID: #116219

Dallas Theological Seminary

Experiment Summary

Timeframe: 11/28/2022 - 12/03/2022

In an effort to increase recurring donor acquisition for a faith-based client during a Giving Tuesday campaign, we wanted to measure the effect of interrupting the donation process with a pop-up asking the donor to become a monthly supporter instead of making only a one-time gift.

This pop-up was set up as an A/B test that would only fire for 50% of donors who were in the process of completing a one-time gift under $100, reiterating the value of becoming a monthly donor both in terms of impact and exclusive benefits.

We tested this same hypothesis with the same client during last year’s Giving Tuesday campaign, and while we saw an encouraging lift in recurring giving, we were unable to validate our results—so we were eager to see how the same concept would perform in a similar setting during this year’s campaign.

Research Question

We believe that interrupting the donation process with a pop-up for donors on Giving Tuesday will improve recurring gift conversion rates by increasing its visibility at the decision-making point and prominently communicating its value.

Design

C: Control

T1: Recurring Gift Popup

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence	Average Gift
C:	Control	3.1%			$0.00
T1:	Recurring Gift Popup	9.8%	210.2%	98.6%	$0.00

This experiment has a required sample size of 107 in order to be valid. Since the experiment had a total sample size of 323, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

0% increase in traffic
× 210.2% increase in conversion rate
× 0% increase in average gift

Key Learnings

Due to an increase in the number of donors (sample size) over last year’s campaign, we were pleased to be able to pull a valid learning from this year’s experiment—with our treatment producing a statistically valid 137% increase in recurring gifts with a 98.4% level of confidence.

We believe that several motivating factors came into play in producing this result:

First, the interrupter pop-up dramatically increased the visibility of the recurring-giving option.
Second, it clearly and thoroughly laid out the value of becoming a recurring donor in terms of impact and exclusive benefits
Third, by creating a mental interruption, it may have given donors an opportunity to reset the mental process of completing their donation (transaction forms being as prevalent as they are in the digital space may cue a certain degree of auto-pilot during the transaction process) and allowed them the time to consider their options.
And finally, the pop-up treatment positioned monthly giving as an opportunity rather than an ask, giving agency to the donor and casting them in the role of someone who is able to do (and get) more than they would by making a one-time gift.

We also observed that the treatment produced a statistically valid increase of 210% in recurring giving among donors using mobile devices, important as we continue to see an increase in mobile giving over time.

These results are a positive indication that a similar treatment may work for other clients in the context of high-urgency campaigns and we will seek to globalize this learning by implementing it for new audiences and new giving contexts, such as content offers or direct appeals.

Experiment Documented by NextAfter

Question about experiment #116219

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.