How a text message impacts campaign giving in a high urgency campaign
Experiment ID: #88886
The Fund for American Studies
Experiment Summary
Ended On: 03/18/2022
During a high urgency campaign TFAS wanted to test using text messaging for the end of their high urgency campaign, Founders Day. Typically, TFAS uses slybroadcast in the final days of their campaign to remind their constituents to give during the campaign. We wanted to determine how effective testing would be by splitting the mobile users list in half. Half would receive the text and half would not. We then looked at downstream conversions during the campaign to see if there was an increase in giving among those who received the text
Research Question
We believe that sending a text message during a high urgency campaign will increase downstream donations among their housefile.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 27965
[id] => 27965
[title] => Treatment_#1.png
[filename] => Treatment_1-411.png
[filesize] => 0
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411.png
[link] => https://www.nextafter.com/experiments/how-a-text-message-impacts-campaign-giving-in-a-high-urgency-campaign/attachment/treatment_1-png-440/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-440
[status] => inherit
[uploaded_to] => 27964
[date] => 2022-04-14 16:30:21
[modified] => 2022-04-14 16:30:21
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 1170
[height] => 2532
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-139x300.png
[medium-width] => 139
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-768x1662.png
[medium_large-width] => 640
[medium_large-height] => 1385
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-473x1024.png
[large-width] => 473
[large-height] => 1024
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-710x1536.png
[1536x1536-width] => 710
[1536x1536-height] => 1536
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-946x2048.png
[2048x2048-width] => 946
[2048x2048-height] => 2048
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411.png
[tp-image-grid-width] => 323
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-23x50.png
[admin-width] => 23
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411.png
[full-width] => 1170
[full-height] => 2532
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-372x805.png
[thumb_372-width] => 372
[thumb_372-height] => 805
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-289x625.png
[single_post_thumbnail-width] => 289
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-15x32.png
[thumb_32_32-width] => 15
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-22x48.png
[thumb_48_48-width] => 22
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-30x64.png
[thumb_64_64-width] => 30
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-59x128.png
[thumb_128_128-width] => 59
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-31x67.png
[thumb_122_67-width] => 31
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-71x153.png
[thumb_153_153-width] => 71
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-75x163.png
[thumb_130_163-width] => 75
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-82x177.png
[thumb_232_177-width] => 82
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-300x650.png
[thumb_272_325-width] => 300
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-160x346.png
[thumb_271_346-width] => 160
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-125x271.png
[thumb_271_271-width] => 125
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-79x172.png
[thumb_272_172-width] => 79
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-213x461.png
[thumb_372_461-width] => 213
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-231x500.png
[thumb_372_500-width] => 231
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-125x270.png
[thumb_385_270-width] => 125
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-111x241.png
[thumb_416_241-width] => 111
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-189x410.png
[thumb_448_410-width] => 189
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-379x820.png
[thumb_448_410_retina-width] => 379
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-177x382.png
[thumb_574_382-width] => 177
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-411-183x395.png
[thumb_776_395-width] => 183
[thumb_776_395-height] => 395
) ) )
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.86% | ||
| T1: | Treatment #1 | 1.1% | 26.0% | 31.1% |
This experiment has a required sample size of 13,386 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The results of this experiment did not validate. There is no statistical difference between those who did or did not receive the text message in the campaign. This means that this text was most likely not a motivator for those to take action during the campaign. Another potential test is to use text messaging as a way to engage users (not just an ask) and see if this type of engagement produces more donations during the campaign.