How point-of-donation upgrading affects donor conversion
Experiment ID: #772
The Heritage Foundation
Experiment Summary
Timeframe: 01/13/2015 - 01/23/2015
The standard Heritage appeal uses a donation form with three buttons as the suggested giving amount. No language is usually included explaining how their gift translates into one of the various membership levels.
As part of new member acquisition campaign, Heritage tested a “point-of-donation” upgrade strategy that placed language on the donation form that explained the benefits associated with the level of their current gift selection and the necessary donation amount required to reach the next level.
Research Question
How important are benefits to new members (new donors) as they decide how much to give to the Heritage Foundation?
Design
C: No Membership Language
Array
(
[0] => Array
(
[ID] => 11576
[id] => 11576
[title] => unnamed-file.C-1.png
[filename] => unnamed-file.C-1.png
[filesize] => 0
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1.png
[link] => https://www.nextafter.com/experiments/how-pushing-for-the-upgrade-affects-donor-conversion/attachment/unnamed-file-c-1-png/
[alt] =>
[author] => 1
[description] =>
[caption] =>
[name] => unnamed-file-c-1-png
[status] => inherit
[uploaded_to] => 11575
[date] => 2021-02-24 10:36:32
[modified] => 2021-02-24 10:36:32
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 1043
[height] => 2720
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-115x300.png
[medium-width] => 115
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-768x2003.png
[medium_large-width] => 640
[medium_large-height] => 1669
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-393x1024.png
[large-width] => 393
[large-height] => 1024
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-589x1536.png
[1536x1536-width] => 589
[1536x1536-height] => 1536
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-785x2048.png
[2048x2048-width] => 785
[2048x2048-height] => 2048
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1.png
[tp-image-grid-width] => 268
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-19x50.png
[admin-width] => 19
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1.png
[full-width] => 1043
[full-height] => 2720
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-372x970.png
[thumb_372-width] => 372
[thumb_372-height] => 970
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-240x625.png
[single_post_thumbnail-width] => 240
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-12x32.png
[thumb_32_32-width] => 12
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-18x48.png
[thumb_48_48-width] => 18
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-25x64.png
[thumb_64_64-width] => 25
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-49x128.png
[thumb_128_128-width] => 49
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-26x67.png
[thumb_122_67-width] => 26
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-59x153.png
[thumb_153_153-width] => 59
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-63x163.png
[thumb_130_163-width] => 63
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-68x177.png
[thumb_232_177-width] => 68
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-249x650.png
[thumb_272_325-width] => 249
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-133x346.png
[thumb_271_346-width] => 133
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-104x271.png
[thumb_271_271-width] => 104
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-66x172.png
[thumb_272_172-width] => 66
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-177x461.png
[thumb_372_461-width] => 177
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-192x500.png
[thumb_372_500-width] => 192
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-104x270.png
[thumb_385_270-width] => 104
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-92x241.png
[thumb_416_241-width] => 92
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-157x410.png
[thumb_448_410-width] => 157
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-314x820.png
[thumb_448_410_retina-width] => 314
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-146x382.png
[thumb_574_382-width] => 146
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.C-1-151x395.png
[thumb_776_395-width] => 151
[thumb_776_395-height] => 395
) ) )
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | No Membership Language | 4.8% | $52.96 | ||
| T1: | Membership Language on the Form | 3.3% | -29.9% | 100.0% | $70.99 |
This experiment has a required sample size of 1,463 in order to be valid. Since the experiment had a total sample size of 9,333, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 29.9% decrease in conversion rate
× 34.0% increase in average gift
6.1% decrease in revenue
Key Learnings
The membership language in the treatment increased the average gift by $18.03 but it caused such a drop in conversion that it actually had a lower total revenue. Future tests may be necessary to understand how specific changes to the point-of-donation language may be able to continue to produce increases in average gift without negatively affecting donation conversion.