How adding a relevant premium in an offer affects giving
Experiment ID: #157957
The Navigators
Experiment Summary
Timeframe: 06/23/2023 - 07/03/2023
Navigators has historically had mixed results with the use of premiums on an offer. We have seen the most success with premiums that are directly correlated with the digital product they’re receiving. In a new offer (an online course offer) we added a relevant premium (the book version of the ebook included) for a gift of $50 or greater. We wanted to test to see if we would get a larger average gift with a relevant premium with this particular offer. Increasing the average gift would positively impact the ROI of the offer, which had been very low.
Research Question
We believe that adding a relevant premium for offer downloaders will achieve an increase in average gift because they would be incentivized to give a larger gift when they receive something in return.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 60173
[id] => 60173
[title] => Control.png
[filename] => Control-3.png
[filesize] => 772751
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3.png
[link] => https://www.nextafter.com/experiments/how-adding-a-relevant-premium-in-an-offer-affects-giving/attachment/control-png-7453/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-7453
[status] => inherit
[uploaded_to] => 53626
[date] => 2024-06-19 14:23:30
[modified] => 2024-06-19 14:23:30
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 2880
[height] => 1820
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-300x190.png
[medium-width] => 300
[medium-height] => 190
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-768x485.png
[medium_large-width] => 640
[medium_large-height] => 404
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-1024x647.png
[large-width] => 640
[large-height] => 404
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-1536x971.png
[1536x1536-width] => 1536
[1536x1536-height] => 971
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-2048x1294.png
[2048x2048-width] => 2048
[2048x2048-height] => 1294
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-50x32.png
[admin-width] => 50
[admin-height] => 32
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3.png
[full-width] => 2880
[full-height] => 1820
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-372x235.png
[thumb_372-width] => 372
[thumb_372-height] => 235
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-980x619.png
[single_post_thumbnail-width] => 980
[single_post_thumbnail-height] => 619
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-32x20.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 20
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-48x30.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 30
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-64x40.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 40
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-128x81.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 81
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-106x67.png
[thumb_122_67-width] => 106
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-153x97.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 97
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-130x82.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 82
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-232x147.png
[thumb_232_177-width] => 232
[thumb_232_177-height] => 147
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-544x344.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 344
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-271x171.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 171
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-271x171.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 171
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-272x172.png
[thumb_272_172-width] => 272
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-372x235.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 235
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-372x235.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 235
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-385x243.png
[thumb_385_270-width] => 385
[thumb_385_270-height] => 243
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-381x241.png
[thumb_416_241-width] => 381
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-448x283.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 283
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-896x566.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 566
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-574x363.png
[thumb_574_382-width] => 574
[thumb_574_382-height] => 363
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-625x395.png
[thumb_776_395-width] => 625
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 60175
[id] => 60175
[title] => Donation_Premium_-_Treatment.png
[filename] => Donation_Premium_-_Treatment.png
[filesize] => 822939
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment.png
[link] => https://www.nextafter.com/experiments/how-adding-a-relevant-premium-in-an-offer-affects-giving/attachment/donation_premium_-_treatment-png-7/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => donation_premium_-_treatment-png-7
[status] => inherit
[uploaded_to] => 53626
[date] => 2024-06-19 14:23:42
[modified] => 2024-06-19 14:23:42
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 2880
[height] => 1948
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-300x203.png
[medium-width] => 300
[medium-height] => 203
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-768x519.png
[medium_large-width] => 640
[medium_large-height] => 433
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-1024x693.png
[large-width] => 640
[large-height] => 433
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-1536x1039.png
[1536x1536-width] => 1536
[1536x1536-height] => 1039
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-2048x1385.png
[2048x2048-width] => 2048
[2048x2048-height] => 1385
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-50x34.png
[admin-width] => 50
[admin-height] => 34
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment.png
[full-width] => 2880
[full-height] => 1948
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-372x252.png
[thumb_372-width] => 372
[thumb_372-height] => 252
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-924x625.png
[single_post_thumbnail-width] => 924
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-32x22.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 22
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-48x32.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 32
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-64x43.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 43
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-128x87.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 87
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-99x67.png
[thumb_122_67-width] => 99
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-153x103.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 103
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-130x88.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 88
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-232x157.png
[thumb_232_177-width] => 232
[thumb_232_177-height] => 157
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-544x368.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 368
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-271x183.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 183
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-271x183.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 183
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-254x172.png
[thumb_272_172-width] => 254
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-372x252.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 252
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-372x252.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 252
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-385x260.png
[thumb_385_270-width] => 385
[thumb_385_270-height] => 260
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-356x241.png
[thumb_416_241-width] => 356
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-448x303.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 303
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-896x606.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 606
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-565x382.png
[thumb_574_382-width] => 565
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-584x395.png
[thumb_776_395-width] => 584
[thumb_776_395-height] => 395
) ) )
T1: Donation Premium - Treatment
Array
(
[0] => Array
(
[ID] => 60173
[id] => 60173
[title] => Control.png
[filename] => Control-3.png
[filesize] => 772751
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3.png
[link] => https://www.nextafter.com/experiments/how-adding-a-relevant-premium-in-an-offer-affects-giving/attachment/control-png-7453/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-7453
[status] => inherit
[uploaded_to] => 53626
[date] => 2024-06-19 14:23:30
[modified] => 2024-06-19 14:23:30
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 2880
[height] => 1820
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-300x190.png
[medium-width] => 300
[medium-height] => 190
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-768x485.png
[medium_large-width] => 640
[medium_large-height] => 404
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-1024x647.png
[large-width] => 640
[large-height] => 404
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-1536x971.png
[1536x1536-width] => 1536
[1536x1536-height] => 971
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-2048x1294.png
[2048x2048-width] => 2048
[2048x2048-height] => 1294
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-50x32.png
[admin-width] => 50
[admin-height] => 32
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3.png
[full-width] => 2880
[full-height] => 1820
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-372x235.png
[thumb_372-width] => 372
[thumb_372-height] => 235
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-980x619.png
[single_post_thumbnail-width] => 980
[single_post_thumbnail-height] => 619
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-32x20.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 20
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-48x30.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 30
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-64x40.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 40
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-128x81.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 81
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-106x67.png
[thumb_122_67-width] => 106
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-153x97.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 97
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-130x82.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 82
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-232x147.png
[thumb_232_177-width] => 232
[thumb_232_177-height] => 147
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-544x344.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 344
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-271x171.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 171
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-271x171.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 171
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-272x172.png
[thumb_272_172-width] => 272
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-372x235.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 235
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-372x235.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 235
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-385x243.png
[thumb_385_270-width] => 385
[thumb_385_270-height] => 243
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-381x241.png
[thumb_416_241-width] => 381
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-448x283.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 283
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-896x566.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 566
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-574x363.png
[thumb_574_382-width] => 574
[thumb_574_382-height] => 363
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-3-625x395.png
[thumb_776_395-width] => 625
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 60175
[id] => 60175
[title] => Donation_Premium_-_Treatment.png
[filename] => Donation_Premium_-_Treatment.png
[filesize] => 822939
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment.png
[link] => https://www.nextafter.com/experiments/how-adding-a-relevant-premium-in-an-offer-affects-giving/attachment/donation_premium_-_treatment-png-7/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => donation_premium_-_treatment-png-7
[status] => inherit
[uploaded_to] => 53626
[date] => 2024-06-19 14:23:42
[modified] => 2024-06-19 14:23:42
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 2880
[height] => 1948
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-300x203.png
[medium-width] => 300
[medium-height] => 203
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-768x519.png
[medium_large-width] => 640
[medium_large-height] => 433
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-1024x693.png
[large-width] => 640
[large-height] => 433
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-1536x1039.png
[1536x1536-width] => 1536
[1536x1536-height] => 1039
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-2048x1385.png
[2048x2048-width] => 2048
[2048x2048-height] => 1385
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-50x34.png
[admin-width] => 50
[admin-height] => 34
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment.png
[full-width] => 2880
[full-height] => 1948
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-372x252.png
[thumb_372-width] => 372
[thumb_372-height] => 252
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-924x625.png
[single_post_thumbnail-width] => 924
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-32x22.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 22
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-48x32.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 32
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-64x43.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 43
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-128x87.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 87
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-99x67.png
[thumb_122_67-width] => 99
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-153x103.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 103
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-130x88.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 88
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-232x157.png
[thumb_232_177-width] => 232
[thumb_232_177-height] => 157
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-544x368.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 368
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-271x183.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 183
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-271x183.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 183
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-254x172.png
[thumb_272_172-width] => 254
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-372x252.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 252
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-372x252.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 252
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-385x260.png
[thumb_385_270-width] => 385
[thumb_385_270-height] => 260
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-356x241.png
[thumb_416_241-width] => 356
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-448x303.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 303
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-896x606.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 606
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-565x382.png
[thumb_574_382-width] => 565
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Donation_Premium_-_Treatment-584x395.png
[thumb_776_395-width] => 584
[thumb_776_395-height] => 395
) ) )
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | $8.89 | $26.67 | ||
| T1: | Donation Premium - Treatment | $2.20 | -75.2% | 95.3% | $19.07 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 65.4% decrease in conversion rate
× 28.5% decrease in average gift
75.2% decrease in revenue
Key Learnings
The key learning from this experiment is that adding a relevant premium for offer downloaders did not lead to an increase in average gift. In fact, the treatment group saw a significant decrease in revenue compared to the control group. This experiment resulted in a valid 75% decrease in revenue with a level of confidence of 95%.
It is important to note that the experiment did not reach the minimum sample size needed for statistical significance. We only had 9 transactions and did not meet the 50 transaction minimum.
A few things to note:
- The offer in it of itself did not perform well. This offer had a high cost per name and did not get a lot of traffic which resulted in a low number of transactions before we had to turn off the experiment.
- In the treatment we wanted to include $50 in the gift array because there is that specific ask for the premium. Unfortunately, the starting amount was significantly lower than the control’s and it’s likely that this lower gift array actually resulted in a lower average gift.
- There was no visual way to call out the premium section. The platform limits our capability and the book on the left hand side is the only place we can add an image.
It’s suggested we retest this with the following adjustments:
- The same gift array for control/treatment
- A visual callout of the premium language so it stands out to the reader (like a box outline around the test)
Based on these findings, it would be recommended to conduct a larger experiment with a sufficient sample size to get a more accurate understanding of the impact of the premium.