How replacing a static image with a video on a Facebook ad affects click-through rates
Experiment ID: #111264
Dallas Theological Seminary
Experiment Summary
Timeframe: 09/18/2022 - 10/18/2022
Many of Dallas Theological Seminary’s Facebook ads for their free online courses are similar, using a color-rich background including the names of the courses and the teachers who present them. In an effort to combat low click through rates, we hypothesized that although viewers were seeing ads for different courses, because these ads looked so similar, it was causing viewers to assume they were seeing an ad for the same course time and time again, eventually prompting them to tune the ads out.
We also wondered if the static ads were inadequately communicating the value one could receive by enrolling in one of Dallas Theological Seminary’s free online courses. We wanted to know if we could increase click-through rates by presenting a video ad that would 1) Catch viewers attention and 2) Better communicate the value of registration.
For this experiment, we changed only the ad media, splitting traffic between he control and treatment over a span of 30 days.
Research Question
We believe that presenting an ad in video format will increase click-through rates for Facebook users by better catching their attention and better communicating the value one receives by signing up.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 54184
[id] => 54184
[title] => Control.png
[filename] => Control-4975.png
[filesize] => 133748
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[link] => https://www.nextafter.com/experiments/how-replacing-a-static-image-with-a-video-on-a-facebook-ad-affects-click-through-rates/attachment/control-png-5845/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-5845
[status] => inherit
[uploaded_to] => 39649
[date] => 2023-08-16 05:33:35
[modified] => 2023-08-16 05:33:35
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 322
[height] => 602
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-160x300.png
[medium-width] => 160
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[medium_large-width] => 322
[medium_large-height] => 602
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[large-width] => 322
[large-height] => 602
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[1536x1536-width] => 322
[1536x1536-height] => 602
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[2048x2048-width] => 322
[2048x2048-height] => 602
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-322x400.png
[ab-block-post-grid-landscape-width] => 322
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-322x600.png
[ab-block-post-grid-square-width] => 322
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[tp-image-grid-width] => 322
[tp-image-grid-height] => 602
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-27x50.png
[admin-width] => 27
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[full-width] => 322
[full-height] => 602
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[thumb_372-width] => 322
[thumb_372-height] => 602
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[single_post_thumbnail-width] => 322
[single_post_thumbnail-height] => 602
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-17x32.png
[thumb_32_32-width] => 17
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-26x48.png
[thumb_48_48-width] => 26
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-34x64.png
[thumb_64_64-width] => 34
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-68x128.png
[thumb_128_128-width] => 68
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-36x67.png
[thumb_122_67-width] => 36
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-82x153.png
[thumb_153_153-width] => 82
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-87x163.png
[thumb_130_163-width] => 87
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-95x177.png
[thumb_232_177-width] => 95
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[thumb_272_325-width] => 322
[thumb_272_325-height] => 602
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-185x346.png
[thumb_271_346-width] => 185
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-145x271.png
[thumb_271_271-width] => 145
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-92x172.png
[thumb_272_172-width] => 92
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-247x461.png
[thumb_372_461-width] => 247
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-267x500.png
[thumb_372_500-width] => 267
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-144x270.png
[thumb_385_270-width] => 144
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-129x241.png
[thumb_416_241-width] => 129
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-219x410.png
[thumb_448_410-width] => 219
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[thumb_448_410_retina-width] => 322
[thumb_448_410_retina-height] => 602
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-204x382.png
[thumb_574_382-width] => 204
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-211x395.png
[thumb_776_395-width] => 211
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 54185
[id] => 54185
[title] => Treatment_#1.png
[filename] => Treatment_1-1413.png
[filesize] => 138452
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[link] => https://www.nextafter.com/experiments/how-replacing-a-static-image-with-a-video-on-a-facebook-ad-affects-click-through-rates/attachment/treatment_1-png-1703/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-1703
[status] => inherit
[uploaded_to] => 39649
[date] => 2023-08-16 05:33:47
[modified] => 2023-08-16 05:33:47
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 320
[height] => 515
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-186x300.png
[medium-width] => 186
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[medium_large-width] => 320
[medium_large-height] => 515
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[large-width] => 320
[large-height] => 515
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[1536x1536-width] => 320
[1536x1536-height] => 515
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[2048x2048-width] => 320
[2048x2048-height] => 515
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-320x400.png
[ab-block-post-grid-landscape-width] => 320
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[ab-block-post-grid-square-width] => 320
[ab-block-post-grid-square-height] => 515
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[tp-image-grid-width] => 320
[tp-image-grid-height] => 515
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-31x50.png
[admin-width] => 31
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[full-width] => 320
[full-height] => 515
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[thumb_372-width] => 320
[thumb_372-height] => 515
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[single_post_thumbnail-width] => 320
[single_post_thumbnail-height] => 515
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-20x32.png
[thumb_32_32-width] => 20
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-30x48.png
[thumb_48_48-width] => 30
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-40x64.png
[thumb_64_64-width] => 40
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-80x128.png
[thumb_128_128-width] => 80
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-42x67.png
[thumb_122_67-width] => 42
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-95x153.png
[thumb_153_153-width] => 95
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-101x163.png
[thumb_130_163-width] => 101
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-110x177.png
[thumb_232_177-width] => 110
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[thumb_272_325-width] => 320
[thumb_272_325-height] => 515
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-215x346.png
[thumb_271_346-width] => 215
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-168x271.png
[thumb_271_271-width] => 168
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-107x172.png
[thumb_272_172-width] => 107
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-286x461.png
[thumb_372_461-width] => 286
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-311x500.png
[thumb_372_500-width] => 311
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-168x270.png
[thumb_385_270-width] => 168
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-150x241.png
[thumb_416_241-width] => 150
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-255x410.png
[thumb_448_410-width] => 255
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[thumb_448_410_retina-width] => 320
[thumb_448_410_retina-height] => 515
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-237x382.png
[thumb_574_382-width] => 237
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-245x395.png
[thumb_776_395-width] => 245
[thumb_776_395-height] => 395
) ) )
T1: Treatment #1
Array
(
[0] => Array
(
[ID] => 54184
[id] => 54184
[title] => Control.png
[filename] => Control-4975.png
[filesize] => 133748
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[link] => https://www.nextafter.com/experiments/how-replacing-a-static-image-with-a-video-on-a-facebook-ad-affects-click-through-rates/attachment/control-png-5845/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-5845
[status] => inherit
[uploaded_to] => 39649
[date] => 2023-08-16 05:33:35
[modified] => 2023-08-16 05:33:35
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 322
[height] => 602
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-160x300.png
[medium-width] => 160
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[medium_large-width] => 322
[medium_large-height] => 602
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[large-width] => 322
[large-height] => 602
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[1536x1536-width] => 322
[1536x1536-height] => 602
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[2048x2048-width] => 322
[2048x2048-height] => 602
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-322x400.png
[ab-block-post-grid-landscape-width] => 322
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-322x600.png
[ab-block-post-grid-square-width] => 322
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[tp-image-grid-width] => 322
[tp-image-grid-height] => 602
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-27x50.png
[admin-width] => 27
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[full-width] => 322
[full-height] => 602
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[thumb_372-width] => 322
[thumb_372-height] => 602
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[single_post_thumbnail-width] => 322
[single_post_thumbnail-height] => 602
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-17x32.png
[thumb_32_32-width] => 17
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-26x48.png
[thumb_48_48-width] => 26
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-34x64.png
[thumb_64_64-width] => 34
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-68x128.png
[thumb_128_128-width] => 68
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-36x67.png
[thumb_122_67-width] => 36
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-82x153.png
[thumb_153_153-width] => 82
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-87x163.png
[thumb_130_163-width] => 87
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-95x177.png
[thumb_232_177-width] => 95
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[thumb_272_325-width] => 322
[thumb_272_325-height] => 602
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-185x346.png
[thumb_271_346-width] => 185
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-145x271.png
[thumb_271_271-width] => 145
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-92x172.png
[thumb_272_172-width] => 92
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-247x461.png
[thumb_372_461-width] => 247
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-267x500.png
[thumb_372_500-width] => 267
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-144x270.png
[thumb_385_270-width] => 144
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-129x241.png
[thumb_416_241-width] => 129
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-219x410.png
[thumb_448_410-width] => 219
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975.png
[thumb_448_410_retina-width] => 322
[thumb_448_410_retina-height] => 602
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-204x382.png
[thumb_574_382-width] => 204
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-4975-211x395.png
[thumb_776_395-width] => 211
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 54185
[id] => 54185
[title] => Treatment_#1.png
[filename] => Treatment_1-1413.png
[filesize] => 138452
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[link] => https://www.nextafter.com/experiments/how-replacing-a-static-image-with-a-video-on-a-facebook-ad-affects-click-through-rates/attachment/treatment_1-png-1703/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-1703
[status] => inherit
[uploaded_to] => 39649
[date] => 2023-08-16 05:33:47
[modified] => 2023-08-16 05:33:47
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 320
[height] => 515
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-186x300.png
[medium-width] => 186
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[medium_large-width] => 320
[medium_large-height] => 515
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[large-width] => 320
[large-height] => 515
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[1536x1536-width] => 320
[1536x1536-height] => 515
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[2048x2048-width] => 320
[2048x2048-height] => 515
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-320x400.png
[ab-block-post-grid-landscape-width] => 320
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[ab-block-post-grid-square-width] => 320
[ab-block-post-grid-square-height] => 515
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[tp-image-grid-width] => 320
[tp-image-grid-height] => 515
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-31x50.png
[admin-width] => 31
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[full-width] => 320
[full-height] => 515
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[thumb_372-width] => 320
[thumb_372-height] => 515
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[single_post_thumbnail-width] => 320
[single_post_thumbnail-height] => 515
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-20x32.png
[thumb_32_32-width] => 20
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-30x48.png
[thumb_48_48-width] => 30
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-40x64.png
[thumb_64_64-width] => 40
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-80x128.png
[thumb_128_128-width] => 80
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-42x67.png
[thumb_122_67-width] => 42
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-95x153.png
[thumb_153_153-width] => 95
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-101x163.png
[thumb_130_163-width] => 101
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-110x177.png
[thumb_232_177-width] => 110
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[thumb_272_325-width] => 320
[thumb_272_325-height] => 515
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-215x346.png
[thumb_271_346-width] => 215
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-168x271.png
[thumb_271_271-width] => 168
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-107x172.png
[thumb_272_172-width] => 107
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-286x461.png
[thumb_372_461-width] => 286
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-311x500.png
[thumb_372_500-width] => 311
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-168x270.png
[thumb_385_270-width] => 168
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-150x241.png
[thumb_416_241-width] => 150
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-255x410.png
[thumb_448_410-width] => 255
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413.png
[thumb_448_410_retina-width] => 320
[thumb_448_410_retina-height] => 515
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-237x382.png
[thumb_574_382-width] => 237
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-1413-245x395.png
[thumb_776_395-width] => 245
[thumb_776_395-height] => 395
) ) )
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.30% | ||
| T1: | Treatment #1 | 0.59% | 93.0% | 100.0% |
This experiment has a required sample size of 4,847 in order to be valid. Since the experiment had a total sample size of 869,386, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
93.0% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running this experiment for 30 days, we observed a 93% LIFT in click-through rates, with a 100% level of confidence.
This experiment validated our hypothesis that a more interactive media format would result in more viewers clicking the ad. We believe that this treatment was successful for two primary reasons:
First, we believe that the new media format broke up the monotony of similarly formatted ads being shown to the same people. A different ad from a familiar organization, we believe, cued viewers that a new offer was being advertised.
Second, we believe that the video allowed viewers to see for themselves the quality of the coursework and the expertise of the teacher (these courses are adapted from on-campus seminary lectures) which increased the perceived value of the offer by improving its credibility and exclusivity, which likely provided more incentive for the viewer to click.
Moving forward, we will attempt to gather more video samples that we can use to promote more courses in their catalog.