How the Freedom Quiz button text on the button promotion impacted petition click rates

Americans for Prosperity

Experiment Summary

Timeframe: 04/03/2019 - 04/05/2019

Americans for Prosperity launched a new quiz, which led to a petition signature page. At the end of the quiz, we wanted to test which button text would generate the most signatures.

Research Question

Which button text at the end of the quiz will generate the highest click rate?

Results

This experiment has a required sample size of 191 in order to be valid. Since the experiment had a total sample size of 256, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

With a 88.5% level of confidence, the treatment text on the button increased petition button click rate by 77.9% over the control.

With our goal being to get as many people from the quiz to the petition (and subsequently the donation page), our learning here is that we should err on the side of “soft asks” on button text. The control message forces the quiz-taker to decide whether or not they want to sign the petition before they’ve ever seen it.

Because the email capture rate is so low across both segments, we wanted to end this experiment quickly to launch an email capture at the front of the quiz experience (instead of at the end of the quiz) to increase overall email capture rates.