How a blog-style cultivation email impacts email engagement and content consumption
Experiment ID: #189874
NextAfter
Experiment Summary
Timeframe: 08/22/2024 - 08/27/2024
In previous experimentation, we’ve learned that shorter emails win higher click rates, but that is for appeal-style emails. We were interested in testing this same concept in a cultivation email to see if the added value would lead to better results.
Since this is a cultivation email, the concept is to see if giving away the value in the email gets more people to consume our content vs. making them take an additional step to click through to the resource. The resource that we did link in both emails was not gated — it was a PDF.
The real objective here isn’t so easily measured, so the results of this test can’t rely totally on quantitative data. Because it’s not a metric-based a/b test, but an operational test we will be tracking email replies to help inform our learning.
Research Question
We believe that a longer-form blog-style cultivation email for email subscribers will achieve more of our content being consumed and better email engagement because it gives the value away without requiring any further action . because it gives the value away without requiring any further action. because it gives the value away without requiring any further action. because it gives the value away without requiring any further action . because it gives the value away without requiring any further action. because it gives the value away without requiring any further action.
Design
C: Version A
Array
(
[0] => Array
(
[ID] => 67911
[id] => 67911
[title] => Version_A.png
[filename] => Version_A-9.png
[filesize] => 106217
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[link] => https://www.nextafter.com/experiments/how-a-blog-style-cultivation-email-impacts-email-engagement-and-content-consumption/attachment/version_a-png-97/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_a-png-97
[status] => inherit
[uploaded_to] => 66019
[date] => 2024-11-05 16:59:14
[modified] => 2024-11-05 16:59:14
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 1120
[height] => 969
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-300x260.png
[medium-width] => 300
[medium-height] => 260
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-768x664.png
[medium_large-width] => 640
[medium_large-height] => 553
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-1024x886.png
[large-width] => 640
[large-height] => 554
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[1536x1536-width] => 1120
[1536x1536-height] => 969
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[2048x2048-width] => 1120
[2048x2048-height] => 969
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-50x43.png
[admin-width] => 50
[admin-height] => 43
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[full-width] => 1120
[full-height] => 969
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-372x322.png
[thumb_372-width] => 372
[thumb_372-height] => 322
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-722x625.png
[single_post_thumbnail-width] => 722
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-32x28.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 28
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-48x42.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 42
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-64x55.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 55
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-128x111.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 111
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-77x67.png
[thumb_122_67-width] => 77
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-153x132.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 132
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-130x112.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 112
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-205x177.png
[thumb_232_177-width] => 205
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-544x471.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 471
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-271x234.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 234
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-271x234.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 234
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-199x172.png
[thumb_272_172-width] => 199
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-372x322.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 322
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-372x322.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 322
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-312x270.png
[thumb_385_270-width] => 312
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-279x241.png
[thumb_416_241-width] => 279
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-448x388.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 388
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-896x775.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 775
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-442x382.png
[thumb_574_382-width] => 442
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-457x395.png
[thumb_776_395-width] => 457
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 67912
[id] => 67912
[title] => Version_B.png
[filename] => Version_B-9.png
[filesize] => 627786
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9.png
[link] => https://www.nextafter.com/experiments/how-a-blog-style-cultivation-email-impacts-email-engagement-and-content-consumption/attachment/version_b-png-89/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_b-png-89
[status] => inherit
[uploaded_to] => 66019
[date] => 2024-11-05 16:59:17
[modified] => 2024-11-05 16:59:17
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 1121
[height] => 3222
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-104x300.png
[medium-width] => 104
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-768x2207.png
[medium_large-width] => 640
[medium_large-height] => 1839
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-356x1024.png
[large-width] => 356
[large-height] => 1024
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-534x1536.png
[1536x1536-width] => 534
[1536x1536-height] => 1536
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-713x2048.png
[2048x2048-width] => 713
[2048x2048-height] => 2048
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-17x50.png
[admin-width] => 17
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9.png
[full-width] => 1121
[full-height] => 3222
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-372x1069.png
[thumb_372-width] => 372
[thumb_372-height] => 1069
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-217x625.png
[single_post_thumbnail-width] => 217
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-11x32.png
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-17x48.png
[thumb_48_48-width] => 17
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-22x64.png
[thumb_64_64-width] => 22
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-45x128.png
[thumb_128_128-width] => 45
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-23x67.png
[thumb_122_67-width] => 23
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-53x153.png
[thumb_153_153-width] => 53
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-57x163.png
[thumb_130_163-width] => 57
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-62x177.png
[thumb_232_177-width] => 62
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-226x650.png
[thumb_272_325-width] => 226
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-120x346.png
[thumb_271_346-width] => 120
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-94x271.png
[thumb_271_271-width] => 94
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-60x172.png
[thumb_272_172-width] => 60
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-160x461.png
[thumb_372_461-width] => 160
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-174x500.png
[thumb_372_500-width] => 174
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-94x270.png
[thumb_385_270-width] => 94
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-84x241.png
[thumb_416_241-width] => 84
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-143x410.png
[thumb_448_410-width] => 143
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-285x820.png
[thumb_448_410_retina-width] => 285
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-133x382.png
[thumb_574_382-width] => 133
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-137x395.png
[thumb_776_395-width] => 137
[thumb_776_395-height] => 395
) ) )
T1: Version B
Array
(
[0] => Array
(
[ID] => 67911
[id] => 67911
[title] => Version_A.png
[filename] => Version_A-9.png
[filesize] => 106217
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[link] => https://www.nextafter.com/experiments/how-a-blog-style-cultivation-email-impacts-email-engagement-and-content-consumption/attachment/version_a-png-97/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_a-png-97
[status] => inherit
[uploaded_to] => 66019
[date] => 2024-11-05 16:59:14
[modified] => 2024-11-05 16:59:14
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 1120
[height] => 969
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-300x260.png
[medium-width] => 300
[medium-height] => 260
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-768x664.png
[medium_large-width] => 640
[medium_large-height] => 553
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-1024x886.png
[large-width] => 640
[large-height] => 554
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[1536x1536-width] => 1120
[1536x1536-height] => 969
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[2048x2048-width] => 1120
[2048x2048-height] => 969
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-50x43.png
[admin-width] => 50
[admin-height] => 43
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9.png
[full-width] => 1120
[full-height] => 969
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-372x322.png
[thumb_372-width] => 372
[thumb_372-height] => 322
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-722x625.png
[single_post_thumbnail-width] => 722
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-32x28.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 28
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-48x42.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 42
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-64x55.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 55
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-128x111.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 111
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-77x67.png
[thumb_122_67-width] => 77
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-153x132.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 132
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-130x112.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 112
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-205x177.png
[thumb_232_177-width] => 205
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-544x471.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 471
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-271x234.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 234
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-271x234.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 234
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-199x172.png
[thumb_272_172-width] => 199
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-372x322.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 322
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-372x322.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 322
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-312x270.png
[thumb_385_270-width] => 312
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-279x241.png
[thumb_416_241-width] => 279
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-448x388.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 388
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-896x775.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 775
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-442x382.png
[thumb_574_382-width] => 442
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_A-9-457x395.png
[thumb_776_395-width] => 457
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 67912
[id] => 67912
[title] => Version_B.png
[filename] => Version_B-9.png
[filesize] => 627786
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9.png
[link] => https://www.nextafter.com/experiments/how-a-blog-style-cultivation-email-impacts-email-engagement-and-content-consumption/attachment/version_b-png-89/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_b-png-89
[status] => inherit
[uploaded_to] => 66019
[date] => 2024-11-05 16:59:17
[modified] => 2024-11-05 16:59:17
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 1121
[height] => 3222
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-104x300.png
[medium-width] => 104
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-768x2207.png
[medium_large-width] => 640
[medium_large-height] => 1839
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-356x1024.png
[large-width] => 356
[large-height] => 1024
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-534x1536.png
[1536x1536-width] => 534
[1536x1536-height] => 1536
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-713x2048.png
[2048x2048-width] => 713
[2048x2048-height] => 2048
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-17x50.png
[admin-width] => 17
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9.png
[full-width] => 1121
[full-height] => 3222
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-372x1069.png
[thumb_372-width] => 372
[thumb_372-height] => 1069
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-217x625.png
[single_post_thumbnail-width] => 217
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-11x32.png
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-17x48.png
[thumb_48_48-width] => 17
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-22x64.png
[thumb_64_64-width] => 22
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-45x128.png
[thumb_128_128-width] => 45
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-23x67.png
[thumb_122_67-width] => 23
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-53x153.png
[thumb_153_153-width] => 53
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-57x163.png
[thumb_130_163-width] => 57
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-62x177.png
[thumb_232_177-width] => 62
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-226x650.png
[thumb_272_325-width] => 226
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-120x346.png
[thumb_271_346-width] => 120
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-94x271.png
[thumb_271_271-width] => 94
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-60x172.png
[thumb_272_172-width] => 60
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-160x461.png
[thumb_372_461-width] => 160
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-174x500.png
[thumb_372_500-width] => 174
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-94x270.png
[thumb_385_270-width] => 94
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-84x241.png
[thumb_416_241-width] => 84
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-143x410.png
[thumb_448_410-width] => 143
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-285x820.png
[thumb_448_410_retina-width] => 285
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-133x382.png
[thumb_574_382-width] => 133
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Version_B-9-137x395.png
[thumb_776_395-width] => 137
[thumb_776_395-height] => 395
) ) )
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Version A | 2.5% | ||
| T1: | Version B | 0.76% | -69.8% | 100.0% |
This experiment has a required sample size of 404 in order to be valid. Since the experiment had a total sample size of 19,851, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
69.8% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The longer treatment email resulted in a 69.8% decrease in clicks. However, the goal of the test wasn’t to get more email clicks. What we are looking to for success in this test is the qualitative data and the email reply rate. Version A had just 1 email reply (.01% reply rate), while Version B had 7 email replies (.07% reply rate). The email replies to Version B all provided positive feedback about the blog-style format. Things to note include:
- They appreciated the tips being given away right in the email
- They liked the plain-text design, formatting, and length of the content
- The advice was timely
- They appreciated not having to complete a form or take another step
Other than that, there were no significant metric differences between the two email versions, so we can conclude that the treatment version did not negatively impact anything.
Because of the positive feedback received regarding the blog-style email, we will begin to implement this email format into the mix for our weekly cultivation emails.