let us help
Subscribe
Canadian Taxpayers Federation

How using a drop down option instead of an open text field on a calculator impacted donor conversion rate

Experiment ID: #82291

Canadian Taxpayers Federation

Experiment Summary

Timeframe: 01/15/2022 - 03/13/2022

As a part of a new campaign to promote a calculator that would help citizens determine what a tax bill on the sale of their home if a proposed Home Equity Tax Bill were to pass, the Canadian Taxpayer Federation wanted to experiment with drop-down fields on the calculator instead of the open text fields in the control.

Research Question

We believe that that using drop-down fields for Home Equity Tax Calculator visitors will achieve an increase in email signup rate and will lift donor conversions overall.

Design

C: Control
Array ( [0] => Array ( [ID] => 40668 [id] => 40668 [title] => Control.jpg [filename] => Control-1688.jpg [filesize] => 404416 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [link] => https://www.nextafter.com/experiments/how-using-a-drop-down-option-instead-of-an-open-text-field-on-a-calculator-impacted-donor-conversion-rate/attachment/control-jpg-1963/ [alt] => [author] => 0 [description] => [caption] => [name] => control-jpg-1963 [status] => inherit [uploaded_to] => 26757 [date] => 2022-10-22 14:43:22 [modified] => 2022-10-22 14:43:22 [menu_order] => 0 [mime_type] => image/jpeg [type] => image [subtype] => jpeg [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 1440 [height] => 1195 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-150x150.jpg [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-300x249.jpg [medium-width] => 300 [medium-height] => 249 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-768x637.jpg [medium_large-width] => 640 [medium_large-height] => 531 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-1024x850.jpg [large-width] => 640 [large-height] => 531 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [1536x1536-width] => 1440 [1536x1536-height] => 1195 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [2048x2048-width] => 1440 [2048x2048-height] => 1195 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-600x400.jpg [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-600x600.jpg [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [tp-image-grid-width] => 700 [tp-image-grid-height] => 581 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-150x150.jpg [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-50x41.jpg [admin-width] => 50 [admin-height] => 41 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [full-width] => 1440 [full-height] => 1195 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-372x309.jpg [thumb_372-width] => 372 [thumb_372-height] => 309 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-753x625.jpg [single_post_thumbnail-width] => 753 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-32x27.jpg [thumb_32_32-width] => 32 [thumb_32_32-height] => 27 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-48x40.jpg [thumb_48_48-width] => 48 [thumb_48_48-height] => 40 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-64x53.jpg [thumb_64_64-width] => 64 [thumb_64_64-height] => 53 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-128x106.jpg [thumb_128_128-width] => 128 [thumb_128_128-height] => 106 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-81x67.jpg [thumb_122_67-width] => 81 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-153x127.jpg [thumb_153_153-width] => 153 [thumb_153_153-height] => 127 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-130x108.jpg [thumb_130_163-width] => 130 [thumb_130_163-height] => 108 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-213x177.jpg [thumb_232_177-width] => 213 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-544x451.jpg [thumb_272_325-width] => 544 [thumb_272_325-height] => 451 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-271x225.jpg [thumb_271_346-width] => 271 [thumb_271_346-height] => 225 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-271x225.jpg [thumb_271_271-width] => 271 [thumb_271_271-height] => 225 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-207x172.jpg [thumb_272_172-width] => 207 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-372x309.jpg [thumb_372_461-width] => 372 [thumb_372_461-height] => 309 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-372x309.jpg [thumb_372_500-width] => 372 [thumb_372_500-height] => 309 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-325x270.jpg [thumb_385_270-width] => 325 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-290x241.jpg [thumb_416_241-width] => 290 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-448x372.jpg [thumb_448_410-width] => 448 [thumb_448_410-height] => 372 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-896x744.jpg [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 744 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-460x382.jpg [thumb_574_382-width] => 460 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-476x395.jpg [thumb_776_395-width] => 476 [thumb_776_395-height] => 395 ) ) [1] => Array ( [ID] => 40670 [id] => 40670 [title] => Dropdown.jpg [filename] => Dropdown-8.jpg [filesize] => 412455 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [link] => https://www.nextafter.com/experiments/how-using-a-drop-down-option-instead-of-an-open-text-field-on-a-calculator-impacted-donor-conversion-rate/attachment/dropdown-jpg-9/ [alt] => [author] => 0 [description] => [caption] => [name] => dropdown-jpg-9 [status] => inherit [uploaded_to] => 26757 [date] => 2022-10-22 14:43:42 [modified] => 2022-10-22 14:43:42 [menu_order] => 0 [mime_type] => image/jpeg [type] => image [subtype] => jpeg [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 1440 [height] => 1195 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-150x150.jpg [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-300x249.jpg [medium-width] => 300 [medium-height] => 249 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-768x637.jpg [medium_large-width] => 640 [medium_large-height] => 531 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-1024x850.jpg [large-width] => 640 [large-height] => 531 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [1536x1536-width] => 1440 [1536x1536-height] => 1195 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [2048x2048-width] => 1440 [2048x2048-height] => 1195 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-600x400.jpg [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-600x600.jpg [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [tp-image-grid-width] => 700 [tp-image-grid-height] => 581 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-150x150.jpg [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-50x41.jpg [admin-width] => 50 [admin-height] => 41 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [full-width] => 1440 [full-height] => 1195 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-372x309.jpg [thumb_372-width] => 372 [thumb_372-height] => 309 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-753x625.jpg [single_post_thumbnail-width] => 753 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-32x27.jpg [thumb_32_32-width] => 32 [thumb_32_32-height] => 27 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-48x40.jpg [thumb_48_48-width] => 48 [thumb_48_48-height] => 40 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-64x53.jpg [thumb_64_64-width] => 64 [thumb_64_64-height] => 53 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-128x106.jpg [thumb_128_128-width] => 128 [thumb_128_128-height] => 106 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-81x67.jpg [thumb_122_67-width] => 81 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-153x127.jpg [thumb_153_153-width] => 153 [thumb_153_153-height] => 127 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-130x108.jpg [thumb_130_163-width] => 130 [thumb_130_163-height] => 108 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-213x177.jpg [thumb_232_177-width] => 213 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-544x451.jpg [thumb_272_325-width] => 544 [thumb_272_325-height] => 451 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-271x225.jpg [thumb_271_346-width] => 271 [thumb_271_346-height] => 225 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-271x225.jpg [thumb_271_271-width] => 271 [thumb_271_271-height] => 225 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-207x172.jpg [thumb_272_172-width] => 207 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-372x309.jpg [thumb_372_461-width] => 372 [thumb_372_461-height] => 309 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-372x309.jpg [thumb_372_500-width] => 372 [thumb_372_500-height] => 309 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-325x270.jpg [thumb_385_270-width] => 325 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-290x241.jpg [thumb_416_241-width] => 290 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-448x372.jpg [thumb_448_410-width] => 448 [thumb_448_410-height] => 372 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-896x744.jpg [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 744 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-460x382.jpg [thumb_574_382-width] => 460 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-476x395.jpg [thumb_776_395-width] => 476 [thumb_776_395-height] => 395 ) ) )
T1: Dropdown
Array ( [0] => Array ( [ID] => 40668 [id] => 40668 [title] => Control.jpg [filename] => Control-1688.jpg [filesize] => 404416 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [link] => https://www.nextafter.com/experiments/how-using-a-drop-down-option-instead-of-an-open-text-field-on-a-calculator-impacted-donor-conversion-rate/attachment/control-jpg-1963/ [alt] => [author] => 0 [description] => [caption] => [name] => control-jpg-1963 [status] => inherit [uploaded_to] => 26757 [date] => 2022-10-22 14:43:22 [modified] => 2022-10-22 14:43:22 [menu_order] => 0 [mime_type] => image/jpeg [type] => image [subtype] => jpeg [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 1440 [height] => 1195 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-150x150.jpg [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-300x249.jpg [medium-width] => 300 [medium-height] => 249 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-768x637.jpg [medium_large-width] => 640 [medium_large-height] => 531 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-1024x850.jpg [large-width] => 640 [large-height] => 531 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [1536x1536-width] => 1440 [1536x1536-height] => 1195 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [2048x2048-width] => 1440 [2048x2048-height] => 1195 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-600x400.jpg [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-600x600.jpg [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [tp-image-grid-width] => 700 [tp-image-grid-height] => 581 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-150x150.jpg [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-50x41.jpg [admin-width] => 50 [admin-height] => 41 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688.jpg [full-width] => 1440 [full-height] => 1195 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-372x309.jpg [thumb_372-width] => 372 [thumb_372-height] => 309 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-753x625.jpg [single_post_thumbnail-width] => 753 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-32x27.jpg [thumb_32_32-width] => 32 [thumb_32_32-height] => 27 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-48x40.jpg [thumb_48_48-width] => 48 [thumb_48_48-height] => 40 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-64x53.jpg [thumb_64_64-width] => 64 [thumb_64_64-height] => 53 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-128x106.jpg [thumb_128_128-width] => 128 [thumb_128_128-height] => 106 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-81x67.jpg [thumb_122_67-width] => 81 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-153x127.jpg [thumb_153_153-width] => 153 [thumb_153_153-height] => 127 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-130x108.jpg [thumb_130_163-width] => 130 [thumb_130_163-height] => 108 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-213x177.jpg [thumb_232_177-width] => 213 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-544x451.jpg [thumb_272_325-width] => 544 [thumb_272_325-height] => 451 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-271x225.jpg [thumb_271_346-width] => 271 [thumb_271_346-height] => 225 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-271x225.jpg [thumb_271_271-width] => 271 [thumb_271_271-height] => 225 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-207x172.jpg [thumb_272_172-width] => 207 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-372x309.jpg [thumb_372_461-width] => 372 [thumb_372_461-height] => 309 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-372x309.jpg [thumb_372_500-width] => 372 [thumb_372_500-height] => 309 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-325x270.jpg [thumb_385_270-width] => 325 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-290x241.jpg [thumb_416_241-width] => 290 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-448x372.jpg [thumb_448_410-width] => 448 [thumb_448_410-height] => 372 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-896x744.jpg [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 744 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-460x382.jpg [thumb_574_382-width] => 460 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-1688-476x395.jpg [thumb_776_395-width] => 476 [thumb_776_395-height] => 395 ) ) [1] => Array ( [ID] => 40670 [id] => 40670 [title] => Dropdown.jpg [filename] => Dropdown-8.jpg [filesize] => 412455 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [link] => https://www.nextafter.com/experiments/how-using-a-drop-down-option-instead-of-an-open-text-field-on-a-calculator-impacted-donor-conversion-rate/attachment/dropdown-jpg-9/ [alt] => [author] => 0 [description] => [caption] => [name] => dropdown-jpg-9 [status] => inherit [uploaded_to] => 26757 [date] => 2022-10-22 14:43:42 [modified] => 2022-10-22 14:43:42 [menu_order] => 0 [mime_type] => image/jpeg [type] => image [subtype] => jpeg [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 1440 [height] => 1195 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-150x150.jpg [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-300x249.jpg [medium-width] => 300 [medium-height] => 249 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-768x637.jpg [medium_large-width] => 640 [medium_large-height] => 531 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-1024x850.jpg [large-width] => 640 [large-height] => 531 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [1536x1536-width] => 1440 [1536x1536-height] => 1195 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [2048x2048-width] => 1440 [2048x2048-height] => 1195 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-600x400.jpg [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-600x600.jpg [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [tp-image-grid-width] => 700 [tp-image-grid-height] => 581 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-150x150.jpg [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-50x41.jpg [admin-width] => 50 [admin-height] => 41 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8.jpg [full-width] => 1440 [full-height] => 1195 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-372x309.jpg [thumb_372-width] => 372 [thumb_372-height] => 309 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-753x625.jpg [single_post_thumbnail-width] => 753 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-32x27.jpg [thumb_32_32-width] => 32 [thumb_32_32-height] => 27 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-48x40.jpg [thumb_48_48-width] => 48 [thumb_48_48-height] => 40 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-64x53.jpg [thumb_64_64-width] => 64 [thumb_64_64-height] => 53 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-128x106.jpg [thumb_128_128-width] => 128 [thumb_128_128-height] => 106 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-81x67.jpg [thumb_122_67-width] => 81 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-153x127.jpg [thumb_153_153-width] => 153 [thumb_153_153-height] => 127 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-130x108.jpg [thumb_130_163-width] => 130 [thumb_130_163-height] => 108 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-213x177.jpg [thumb_232_177-width] => 213 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-544x451.jpg [thumb_272_325-width] => 544 [thumb_272_325-height] => 451 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-271x225.jpg [thumb_271_346-width] => 271 [thumb_271_346-height] => 225 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-271x225.jpg [thumb_271_271-width] => 271 [thumb_271_271-height] => 225 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-207x172.jpg [thumb_272_172-width] => 207 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-372x309.jpg [thumb_372_461-width] => 372 [thumb_372_461-height] => 309 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-372x309.jpg [thumb_372_500-width] => 372 [thumb_372_500-height] => 309 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-325x270.jpg [thumb_385_270-width] => 325 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-290x241.jpg [thumb_416_241-width] => 290 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-448x372.jpg [thumb_448_410-width] => 448 [thumb_448_410-height] => 372 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-896x744.jpg [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 744 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-460x382.jpg [thumb_574_382-width] => 460 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Dropdown-8-476x395.jpg [thumb_776_395-width] => 476 [thumb_776_395-height] => 395 ) ) )

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 1.1%$0.00
T1: Dropdown 0.46%-58.0% 99.4%$0.00

This experiment has a required sample size of 1,461 in order to be valid. Since the experiment had a total sample size of 5,631, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 58.0% decrease in conversion rate
× 0% increase in average gift

Key Learnings

Although we observed a valid increase of +13.9% (LoC: 100%) in email signup rate, we noted that the visitors coming through the drop-down treatment experience were 58% less likely to make a donation.

Our hypothesis is that the drop-downs, which used a slightly different calculation may have been showing a lower tax bill forecast than when visitors entered their own information specifically.

Therefore, we decided to turn off this experience and set up a new experiment designed to mitigate anxiety (psychological resistance to filling out the calculator form) to see if we could drive the higher email signup rates experienced within the drop-down experience, but keep the higher donor conversion rate values found in this control experience.

Notable changes to other metrics due to the experiment:

  • Email Signup Rate experienced a 13.9% increase with a 100% level of confidence.

Observations of the impact on other visitor segments in the experiment:

  • Desktop visitors had a 63.6% decrease in donations with a 99% level of confidence.
  • Returning visitors had a 71.6% decrease in donations with a 100% level of confidence.
  • Email visitors had a 60.0% decrease in donations with a 99% level of confidence.
  • Female visitors had a 100.0% decrease in donations with a 96% level of confidence.
  • Mobile visitors had a 63.6% decrease in donations with an 88% level of confidence.
  • Organic Search visitors had a 100.0% decrease in donations with an 86% level of confidence.

Experiment Documented by Greg Colunga
Greg Colunga is Chief Operating Office at NextAfter.

Question about experiment #82291

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.