Save the Children

How increasing the perceived security on a donation page impacts donor conversion

Experiment ID: #67053

Save the Children

Experiment Summary

Timeframe: 06/07/2021 - 07/21/2021

There are many elements of friction on a donation page. One of the most significant areas where friction can be found on these pages is the credit card section. Prospective donors are asked to provide their financial information in this section which can cause anxiety for people during the giving process. We hypothesized that visually telling the donor that their information was secure may impact their liklihood to complete the donation process. To test into this, Save the Children added a padlock image and border around the credit card section to visually communicate to people that their information is safe, it’s secure.

Research Question

Will adding visual elements to the credit card section increase perceived security and ultimately donor conversion?

Design

C: Control

T1: With secure visual elements

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence
C:	Control	16.1%
T1:	With secure visual elements	16.2%	1.1%	27.6%

This experiment has a required sample size of 346,483 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

Overall transactions, there was no impact on donor conversion. It is important to keep in mind that this experiment was executed on the primary donation page where differing motivations from all channels types are pushing traffic. While there wasn’t any impact – positive or negative – on overall donor conversion, specific segments did see a significant impact.

On mobile devices, which makes up 60% of Save the Children’s traffic, the perceived security feature increased donor conversion by 10% at a 96% level of confidence. As we know, the donor experience on mobile devices is very different from desktop. With the smaller device, everything is a bit more prominent – which would make the visual elements added to the form even more apparent and easing the minds of people as they were giving on their phones. These results also bring up an interesting hypothesis that mobile users may be more concerned about the safety of their information than desktop users. This sounds counterintuitive, but so much is being done now on commerce sites to make purchasing so much easier that people normally don’t even have to think about that kind of stuff. The visual reminder may have brought this to their attention and caused them to feel more at ease, confirming in their mind that this was, in fact, secure.

We also saw an increase in giving for returning visitors. There was a 13% increase in donor conversion at a 92% LoC. While this didn’t validate for this segment, the results were directional and indicate that they will, more than not, have a positive response to the higher perceived security. It is interesting that new visitors weren’t impacted in the same way. Without knowing if these return visitors were previous donors or not, what we can learn from this is that the visual security elements may have been the motivating factor to tip the balance for returning visitors in their decision to actually make a gift.

Because there is no negative impact on any segment or channel through this experiment, and some segments will – with confidence – see an increase in donor conversion, we propose rolling this feature out among all donation pages.

Experiment Documented by Courtney Gaines

Courtney Gaines is Vice President at NextAfter.

Question about experiment #67053

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.