How the level of detail in the messaging impacts the email response rate
Experiment ID: #6095
Illinois Policy Institute
Experiment Summary
Timeframe: 01/13/2017 - 01/27/2017
At the start of the Illinois legislative session, Illinois Policy wanted to test an appeal from the two parts of their organization, their 501c3 and their 501c4. Both of the appeals would center around the upcoming session and the need to apply pressure to legislators like Michael Madigan. The primary difference would be in the approach that the different arms would take in applying the pressure. The C3 would be about the general mission to create greater prosperity in Illinois and the C4 would explain the specific marketing campaign they were taking on to educate the public.
Research Question
Will an appeal from the C3 arm be more effective than an appeal from the C4 arm?
Design
C: C3 Appeal
Array
(
[0] => Array
(
[ID] => 13371
[id] => 13371
[title] => unnamed-file.c-3.jpg
[filename] => unnamed-file.c-3.jpg
[filesize] => 0
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[link] => https://www.nextafter.com/experiments/how-the-level-of-detail-in-the-messaging-impacts-the-email-response-rate/attachment/unnamed-file-c-3-jpg/
[alt] =>
[author] => 1
[description] =>
[caption] =>
[name] => unnamed-file-c-3-jpg
[status] => inherit
[uploaded_to] => 13370
[date] => 2021-02-25 06:03:00
[modified] => 2021-02-25 06:03:00
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 800
[height] => 1014
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-237x300.jpg
[medium-width] => 237
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-768x973.jpg
[medium_large-width] => 640
[medium_large-height] => 811
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[large-width] => 640
[large-height] => 811
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[1536x1536-width] => 800
[1536x1536-height] => 1014
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[2048x2048-width] => 800
[2048x2048-height] => 1014
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-600x400.jpg
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-600x600.jpg
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[tp-image-grid-width] => 552
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-39x50.jpg
[admin-width] => 39
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[full-width] => 800
[full-height] => 1014
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-372x472.jpg
[thumb_372-width] => 372
[thumb_372-height] => 472
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-493x625.jpg
[single_post_thumbnail-width] => 493
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-25x32.jpg
[thumb_32_32-width] => 25
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-38x48.jpg
[thumb_48_48-width] => 38
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-50x64.jpg
[thumb_64_64-width] => 50
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-101x128.jpg
[thumb_128_128-width] => 101
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-53x67.jpg
[thumb_122_67-width] => 53
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-121x153.jpg
[thumb_153_153-width] => 121
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-130x163.jpg
[thumb_130_163-width] => 130
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-140x177.jpg
[thumb_232_177-width] => 140
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-513x650.jpg
[thumb_272_325-width] => 513
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-271x343.jpg
[thumb_271_346-width] => 271
[thumb_271_346-height] => 343
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-214x271.jpg
[thumb_271_271-width] => 214
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-136x172.jpg
[thumb_272_172-width] => 136
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-364x461.jpg
[thumb_372_461-width] => 364
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-372x472.jpg
[thumb_372_500-width] => 372
[thumb_372_500-height] => 472
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-213x270.jpg
[thumb_385_270-width] => 213
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-190x241.jpg
[thumb_416_241-width] => 190
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-323x410.jpg
[thumb_448_410-width] => 323
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-647x820.jpg
[thumb_448_410_retina-width] => 647
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-301x382.jpg
[thumb_574_382-width] => 301
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-312x395.jpg
[thumb_776_395-width] => 312
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 13372
[id] => 13372
[title] => t.jpg
[filename] => t.jpg
[filesize] => 0
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[link] => https://www.nextafter.com/experiments/how-the-level-of-detail-in-the-messaging-impacts-the-email-response-rate/attachment/t-jpg/
[alt] =>
[author] => 1
[description] =>
[caption] =>
[name] => t-jpg
[status] => inherit
[uploaded_to] => 13370
[date] => 2021-02-25 06:03:07
[modified] => 2021-02-25 06:03:07
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 800
[height] => 1165
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-206x300.jpg
[medium-width] => 206
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-768x1118.jpg
[medium_large-width] => 640
[medium_large-height] => 932
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-703x1024.jpg
[large-width] => 640
[large-height] => 932
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[1536x1536-width] => 800
[1536x1536-height] => 1165
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[2048x2048-width] => 800
[2048x2048-height] => 1165
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-600x400.jpg
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-600x600.jpg
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[tp-image-grid-width] => 481
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-34x50.jpg
[admin-width] => 34
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[full-width] => 800
[full-height] => 1165
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-372x542.jpg
[thumb_372-width] => 372
[thumb_372-height] => 542
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-429x625.jpg
[single_post_thumbnail-width] => 429
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-22x32.jpg
[thumb_32_32-width] => 22
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-33x48.jpg
[thumb_48_48-width] => 33
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-44x64.jpg
[thumb_64_64-width] => 44
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-88x128.jpg
[thumb_128_128-width] => 88
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-46x67.jpg
[thumb_122_67-width] => 46
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-105x153.jpg
[thumb_153_153-width] => 105
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-112x163.jpg
[thumb_130_163-width] => 112
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-122x177.jpg
[thumb_232_177-width] => 122
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-446x650.jpg
[thumb_272_325-width] => 446
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-238x346.jpg
[thumb_271_346-width] => 238
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-186x271.jpg
[thumb_271_271-width] => 186
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-118x172.jpg
[thumb_272_172-width] => 118
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-317x461.jpg
[thumb_372_461-width] => 317
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-343x500.jpg
[thumb_372_500-width] => 343
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-185x270.jpg
[thumb_385_270-width] => 185
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-165x241.jpg
[thumb_416_241-width] => 165
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-282x410.jpg
[thumb_448_410-width] => 282
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-563x820.jpg
[thumb_448_410_retina-width] => 563
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-262x382.jpg
[thumb_574_382-width] => 262
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-271x395.jpg
[thumb_776_395-width] => 271
[thumb_776_395-height] => 395
) ) )
T1: C4 Appeal
Array
(
[0] => Array
(
[ID] => 13371
[id] => 13371
[title] => unnamed-file.c-3.jpg
[filename] => unnamed-file.c-3.jpg
[filesize] => 0
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[link] => https://www.nextafter.com/experiments/how-the-level-of-detail-in-the-messaging-impacts-the-email-response-rate/attachment/unnamed-file-c-3-jpg/
[alt] =>
[author] => 1
[description] =>
[caption] =>
[name] => unnamed-file-c-3-jpg
[status] => inherit
[uploaded_to] => 13370
[date] => 2021-02-25 06:03:00
[modified] => 2021-02-25 06:03:00
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 800
[height] => 1014
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-237x300.jpg
[medium-width] => 237
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-768x973.jpg
[medium_large-width] => 640
[medium_large-height] => 811
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[large-width] => 640
[large-height] => 811
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[1536x1536-width] => 800
[1536x1536-height] => 1014
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[2048x2048-width] => 800
[2048x2048-height] => 1014
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-600x400.jpg
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-600x600.jpg
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[tp-image-grid-width] => 552
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-39x50.jpg
[admin-width] => 39
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3.jpg
[full-width] => 800
[full-height] => 1014
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-372x472.jpg
[thumb_372-width] => 372
[thumb_372-height] => 472
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-493x625.jpg
[single_post_thumbnail-width] => 493
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-25x32.jpg
[thumb_32_32-width] => 25
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-38x48.jpg
[thumb_48_48-width] => 38
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-50x64.jpg
[thumb_64_64-width] => 50
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-101x128.jpg
[thumb_128_128-width] => 101
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-53x67.jpg
[thumb_122_67-width] => 53
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-121x153.jpg
[thumb_153_153-width] => 121
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-130x163.jpg
[thumb_130_163-width] => 130
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-140x177.jpg
[thumb_232_177-width] => 140
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-513x650.jpg
[thumb_272_325-width] => 513
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-271x343.jpg
[thumb_271_346-width] => 271
[thumb_271_346-height] => 343
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-214x271.jpg
[thumb_271_271-width] => 214
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-136x172.jpg
[thumb_272_172-width] => 136
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-364x461.jpg
[thumb_372_461-width] => 364
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-372x472.jpg
[thumb_372_500-width] => 372
[thumb_372_500-height] => 472
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-213x270.jpg
[thumb_385_270-width] => 213
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-190x241.jpg
[thumb_416_241-width] => 190
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-323x410.jpg
[thumb_448_410-width] => 323
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-647x820.jpg
[thumb_448_410_retina-width] => 647
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-301x382.jpg
[thumb_574_382-width] => 301
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/unnamed-file.c-3-312x395.jpg
[thumb_776_395-width] => 312
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 13372
[id] => 13372
[title] => t.jpg
[filename] => t.jpg
[filesize] => 0
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[link] => https://www.nextafter.com/experiments/how-the-level-of-detail-in-the-messaging-impacts-the-email-response-rate/attachment/t-jpg/
[alt] =>
[author] => 1
[description] =>
[caption] =>
[name] => t-jpg
[status] => inherit
[uploaded_to] => 13370
[date] => 2021-02-25 06:03:07
[modified] => 2021-02-25 06:03:07
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 800
[height] => 1165
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-206x300.jpg
[medium-width] => 206
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-768x1118.jpg
[medium_large-width] => 640
[medium_large-height] => 932
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-703x1024.jpg
[large-width] => 640
[large-height] => 932
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[1536x1536-width] => 800
[1536x1536-height] => 1165
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[2048x2048-width] => 800
[2048x2048-height] => 1165
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-600x400.jpg
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-600x600.jpg
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[tp-image-grid-width] => 481
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-34x50.jpg
[admin-width] => 34
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t.jpg
[full-width] => 800
[full-height] => 1165
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-372x542.jpg
[thumb_372-width] => 372
[thumb_372-height] => 542
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-429x625.jpg
[single_post_thumbnail-width] => 429
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-22x32.jpg
[thumb_32_32-width] => 22
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-33x48.jpg
[thumb_48_48-width] => 33
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-44x64.jpg
[thumb_64_64-width] => 44
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-88x128.jpg
[thumb_128_128-width] => 88
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-46x67.jpg
[thumb_122_67-width] => 46
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-105x153.jpg
[thumb_153_153-width] => 105
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-112x163.jpg
[thumb_130_163-width] => 112
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-122x177.jpg
[thumb_232_177-width] => 122
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-446x650.jpg
[thumb_272_325-width] => 446
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-238x346.jpg
[thumb_271_346-width] => 238
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-186x271.jpg
[thumb_271_271-width] => 186
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-118x172.jpg
[thumb_272_172-width] => 118
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-317x461.jpg
[thumb_372_461-width] => 317
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-343x500.jpg
[thumb_372_500-width] => 343
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-185x270.jpg
[thumb_385_270-width] => 185
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-165x241.jpg
[thumb_416_241-width] => 165
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-282x410.jpg
[thumb_448_410-width] => 282
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-563x820.jpg
[thumb_448_410_retina-width] => 563
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-262x382.jpg
[thumb_574_382-width] => 262
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/t-271x395.jpg
[thumb_776_395-width] => 271
[thumb_776_395-height] => 395
) ) )
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | C3 Appeal | 0.04% | ||
| T1: | C4 Appeal | 0.01% | -84.8% | 99.4% |
This experiment has a required sample size of 10,706 in order to be valid. Since the experiment had a total sample size of 121,193, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 84.8% decrease in conversion rate
× 25.3% decrease in average gift
88.7% decrease in revenue
Key Learnings
When the email was first sent to the initial 50% of the subscriber base, we saw a 20% increase in the percentage of clicks coming from the C4 appeal but it was not a statistically significant amount. What this means is that given the low level of confidence (77%), we cannot make many assumptions based upon this lift. Unfortunately, given the current Mailchimp setup, we were not able to actively monitor the gifts that were coming from the individual emails. As a result, the C4 appeal was select since it had the highest click through rate (even if it wasn’t statistically significant.)
Since these two emails went to different landing pages, we were able to analyze the donation results once the campaign had finished, even though the C4 appeal maintained a slightly higher clickthrough rate than the control, the response rate (or percentage of subscribers that made a gift) was significantly higher for the C3 appeal. There could be several factors that led to this result:
- The C3 arm of the organization is a tax-deductible contribution. This might have factored into a subscribers willingness to give.
- The C3 appeal focused more on the primary value proposition of the organization (meaning the big picture) while the C4 focused on a product-level value proposition (meaning a specific thing they were going to do). This may imply that donors are not as likely to respond when the messaging dives into the tactical approach instead of the high level reasoning for the work.
These hypotheses would need be isolated and tested in future appeals to determine which led to the overall results of the campaign.