Resources Home

The only way to truly know what works to increase donation and grow generosity is to test. And when I say “test”, I don’t mean:

• “We tried that last year, and it didn’t work.”
• “We started doing that in our emails, and our open rates have been better.”
• “We did that on our donation page, and our donors say they love it.”
• “We can’t do that – our board doesn’t like it.”

Those phrases, although common to hear from fundraisers and marketers – don’t put the donor’s behaviors first.

If you try something new and see a different result, how do you know it wasn’t a fluke or influenced by something else?

If you ask your donors for their opinion, how do you know their words match their actions?

And how do you know that your board’s opinion isn’t steering your results in the wrong direction?

A/B testing and optimization is the only way to cut through the noise and allow your donors to show you what actually works to inspire them to give more.

Over the past 10 years at NextAfter, we’ve conducted 3,500+ online fundraising experiments. And the goal has always been to decode what works in fundraising – and equip as many nonprofits as possible with what we’re learning.

So today, I want to show you 15 experiments (or a/b tests) that the NextAfter staff has found to be meaningful over the past 10 years of testing. Some of them led to big increases in results. Others may have led to a decrease, but they also led to important learnings about donor motivations.

10 Favorite A/B Tests from 10 Years of Fundraising Optimization

EXPERIMENT ID: #93844

How Using a Digital Offer Instead of a Physical Offer Increases Email Acquisition

Hypothesis

We believe that offering a digital download for a Facebook audience will achieve a better email conversion rate.

Experiment Summary

In this experiment, the organization wanted to know if the physical resource they were offering was the most effective. Physical offers – like a book, DVD, or anything you must put in the mail – have a significant cost to them. So even if changing the offer to something digital made no difference in performance, it would be a win because of the cost savings.

The Results

This organization was able to achieve a massive 558% increase in email conversion rates, with a 100% level of confidence

Moreover, the cost to acquire an email address was 67% lower than the digital premium. We believe this positive result is due to a combination of factors:

• By offering up immediate gratification through a digital download, versus potentially waiting up to a month to receive your resource in the mail, the audience was much more likely to sign up to receive it.
• By not requiring a full mailing address for the digital version, we removed some friction in the sign-up process.
• The organization also saved the costs of printing, shipping, and labor of a physical premium.

EXPERIMENT ID: 90849

How asking for less than a donor’s last gift impacts email donations

Hypothesis

We believe that asking the donor for less than their last gift will achieve an increase in conversion rates.

Experiment Summary

The organization wanted to test a relatively new concept known as the downgraded ask. For this experiment, they created a chart of standards based on the donors’ previous highest contribution where they would ask for a certain number of donors to give a specific amount.

For the treatment version of this ask, they asked for a level below what the donor had previously given and added language acknowledging that they were being asked for less. This was then tested across an email campaign that included 3 separate emails, with a resend 3 days after the initial send of each. The organization had also never tested including a donor’s previous last gift, so this was a completely new approach for their email file.

The Results

From a donor conversion standpoint, there is a 70% increase in conversion rates. When looking at the results from a revenue perspective, the positive trend continues, with an 85% increase in revenue per visitor and a level of confidence of 100% due to the larger number of gifts made to the treatment version.

• There was also a 26% increase in click-through rates and a 100% level of confidence.
• And finally, of those that clicked the links to donate, 35% of them were more likely to convert to donors with the downgrade treatment version.

EXPERIMENT ID: 19407

How a Clear Call to Action Impacted Monthly Donor Conversion on a Donation Page

Hypothesis

Will a stripped-down value proposition with a clear impact on CTA increase the number of monthly donors acquired on the donation page?

Experiment Summary

This organization has used the same value propositional language for its monthly donor acquisition page for a long time. Every January, they would send out a series of appeals seeking to raise new monthly donors, donors called “Caring Partners.”

As we prepared for the January 2019 campaign, we revisited the donation page and realized that the copy was full of organizational language that may add friction. We wondered if anyone really wanted to become a “caring partner” or even knew what that phrase meant. We also saw that we had a copy dedicated to selling a list of benefits to becoming a monthly donor and wondered if those benefits really mattered. Did people become monthly donors because they wanted a “dedicated team member to serve them?” We stripped out these sections of the page and radically redesigned the copy. We boiled down the value proposition into one key statement: Save babies 365 days a year by becoming a monthly donor. We kept the copy focused on the donor and the impact of their gift of $1/day.

We linked it to a concrete number of lives saved by monthly donors since 2008. We broke out the impact of the monthly gift using bullet points. We did not set up a straight AB test, however, we had historic data from the page we used the prior campaign that we could use to compare to the newly redesigned page.

The Results

With this test, we transformed the monthly donor program overnight. Based on this new value proposition, we began testing other elements and CTAs to include that language, “Save babies from abortion 365 days a year” elsewhere, which grew our monthly giving program 3x in 3 years.

The new page performed far and above the old one. It was so successful, that we kept using it in a wide variety of campaign offers which is why the traffic was so much higher than the control page. It is important to note that the variation had a conversion rate of around 15% during the January campaign, the lower conversion rate reflects the average of all visitors to the page between Jan-August. 

EXPERIMENT ID: 24167

How additional cultivation impacts online giving

Hypothesis

Will an increase in cultivation emails have any impact on a donor’s likelihood to give?

Experiment Summary

During a data analysis performed for an organization at the beginning of the calendar year, we discovered a handful of trends related to the email file that was concerning:

• 3 out of 4 ADF subscribers had not opened an email within the last six months.
• New and existing donors were twice as likely to disengage as non-donor subscribers.
• The organization was sending twice as many solicitation emails as it was cultivation and stewardship-focused emails.
• The subscribers were being asked twice as much as they were given back.

While the digital fundraising program had been growing, these trends illustrated a potential problem soon if they were not corrected. Given the ambitious fundraising goals, we couldn’t reduce the number of solicitations. So, instead, we decided to test out an additional cultivation series that would increase the number of emails sent to the file. The additional emails would focus on educating and feeding back into the various segments. There would be no additional solicitations.

The Results

After running for a full six months, we observed a statistically significant 13.1% increase in the likelihood for a donor to give a gift online. Combined with the 25% increase in average gift, this resulted in a 41.5% increase in online revenue from the donors.

If these cultivation series were rolled out to the entire file and run for an entire year, the projected impact would be an additional $262K raised online without sending any additional solicitations. We also observed a 43% increase in online engagement for all segments.

While the emphasis of this experiment was to measure the impact of online giving and engagement, we also observed several other trends: –

• The impact of the cultivation did not fade over time. We observed the same statistically significant increase in the likelihood to give and revenue when we compared the first three months of giving to the second three months.
• The additional cultivation did not significantly impact a donor’s likelihood to give across all channels, but we did see an increase in the amount they gave by 30% regardless of the channel with a 99% level of confidence.
• It should be noted that this lift did not reach the required sample size so this should be considered a directional lift.
• The additional cultivation did not increase the likelihood of a non-donor making their first gift. However, it did increase their average revenue per donor by 50% in the 6-month period.

EXPERIMENT ID: 61736

How does having a text-based donation page and email vs video affects donor conversion rate

Hypothesis

We believe that a text-based donation page for those that visit the page will achieve a higher conversion rate and more revenue.

Experiment Summary

This experiment ran a donation page with a video versus a text-based donation page with copy that matched the video content. The goal was to test to see if we could get more eyes on what was being communicated on the donation page and increase motivation to make a gift by transcribing the video.

The Results

The text-based donation page led to a 527% increase in conversions! The results here may be surprising, but we can hypothesize that more people are apt to read copy on a donation page rather than to click and watch a three-minute video. This experiment is a great example of why it is so important to test, test, test. What works and what doesn’t to increase donations may surprise you!

EXPERIMENT ID: 42740

How does mentioning the last gift date affect lapsed donor reactivation?

Hypothesis

Does simply including a lapsed donor’s last gift date gently nudge them to give again?

Experiment Summary

During the calendar year-end campaign, this organization tested mentioning the lapsed donor’s last gift date. One version was the same copy as the other campaigns. The test version included the last gift date and last gift amount for each lapsed donor.

The purpose was to remind them that it had been some time since they last gave. We only included donors who had not given in the last 5 years. This was tested over a series of three emails, one campaign email, one final day campaign email, and then a forward of that email from the primary signer’s secretary.

The Results

This experiment required sample size of 758, a total sample size of 2,673, and an LoC above 95%. The last date including copy had a relative difference of 247.2% in the conversion rate (a rate of 1.2% vs. 0.36%).

We learned that recognizing a donor’s last gift, even if they have lapsed, impacts the rate at which they return to give again. Fundraisers should never forget to acknowledge and express gratitude for the generosity of their donors, no matter how long it has been since a gift was given.

EXPERIMENT ID: 36100

Does adding clarity and specificity increase newsletter signups?

Hypothesis

Will adding clarity and specificity to a newsletter signup offer increase the number of email signups?

Experiment Summary

In looking at ways to acquire more email addresses through a global footer across our organization’s website, we looked at the current value prop offered for our email newsletter signup. We saw an opportunity to provide additional language to increase specificity and exclusivity to get more people over to the newsletter subscription page and connected with updates from Buckner!

The Results

Simply by revising the headline and adding 4-5 sentences that highlighted the true value proposition to this newsletter signup footer, we were able to increase the conversion rate by 276%.

Rather than asking people to sign up for a “newsletter”, we unpacked what all types of communications would be sent and how often they could expect to receive messages while also increasing the perceived exclusivity of the content being offered and emphasizing the overall value that would be provided to the reader in exchange for providing their email address.

EXPERIMENT ID: 81122

How a yes/no pop-up with value proposition language impacts donor conversion during a high urgency campaign.

Hypothesis

We believe that adding a value prop-based pop-up for site visitors will achieve a higher donor conversion rate during the calendar year-end.

Experiment Summary

In this experiment, the organization wanted to try exit intent pop-ups in lieu of a homepage takeover for the last few days of the campaign. There was concern that constituents who come to their site for content, may not want to contribute to a campaign. So, we decided to test the language in the yes/no exit intent pop-up.

The control asked the site visitor if they wanted to donate to the organization to support their ongoing efforts. While the treatment would have value proposition-based language, asking the site visitor if they want to bring spiritual guidance to those looking for truth.

The Results

The results indicate that the site visitors were more likely to give to their calendar year-end campaign when presented with a yes/no pop-up that identified the organization’s key value proposition.

Without having any ask attached to the copy, that audience segment produced a 3.1% conversion rate resulting in a 125% increase in donor conversion rate with a level of confidence of 94.8%.

This tells us that audience responds to their core mission and cause over just being asked for a donation. It is recommended that this concept is retested outside of calendar year-end to determine if this is also true in a different context.

EXPERIMENT ID: 18201

How location-based imagery affects the click-through rate of a Facebook Ad campaign

Hypothesis

Will location-specific imagery lead to greater landing page click-throughs in a geo-targeted ad campaign?

Experiment Summary

We were testing to see if using a geographic image of the individual’s current city instead of the typical action shot imagery used for courses could lead to an increased click-through rate.

The Results

There was a 60% increase in revenue for this experiment. We are all aware that adding a

personal touch to marketing can drive additional engagement, but who would’ve thought that geo-fenced imagery could drive such growth.

The hypothesis was that using geographic imagery for an online course would generate a decrease in revenue because technically the imagery is unrelated. This just shows the importance of testing, and not assuming our perspective and experiences are the best indicators of what can or will be successful.

Learn how to set up, run, and document a/b tests in this free workbook

In this free a/b testing guide you’ll get to walk through a/b testing methodology, think critically about how to optimize your online fundraising, and develop your own a/b test that you can run right away.

By the end of the workbook, you’ll be equipped to begin testing for your organization so that you can implement strategies that you know will reach more donors and raise more money.

Learn more and get a free copy of the Ultimate A/B Testing Guide »