FamilyLife

How a radical redesign of an email appeal and donation page affected donor conversion rate

Experiment ID: #6988

FamilyLife

Experiment Summary

Timeframe: 07/14/2017 - 07/21/2017

FamilyLife had a mature email fundraising program that relied heavily on third-party stories with large images. wanted to test an entirely new approach to their email fundraising that integrated a more personal tone and removed possible sources of friction from the “mental conversation” provoked by the email.

The control email used a template with a large FamilyLife logo at the top, followed by an image, a pull quote and a donate button, before any actual email copy. The email copy made an offer to provide a scholarship for a couple like “Fred and Melissa”, even though that couple had not been introduced yet. This presented clarity issues—who are Fred and Melissa? The couple in the picture? The call to action never really requested a gift, it just suggested a gift by linking text telling the reader what their gift would do. Finally, the email had a donate button at the bottom as well. This email led to a landing page that had the headline “Scholarships for Hardworking Pastor Couples”, which didn’t address the reader. That copy gave more information about Fred and Melissa, and then led into a separate call to action section (visually separated by a green box) that was disconnected from the pastor-centric message: “Help Families Stay Together”. This extra section then led to a donation form that had all the fields stacked on top of each other.

The FamilyLife team started with the email, developing a treatment that stripped out all the visual elements to reduce visual friction and resemble an email that their founder, Dennis Rainey, might actually write. Then they took a new approach with the tone, making it very personal. The treatment copy talked about his wife, their history serving pastors, and the challenges they’ve seen firsthand. The ask was vey clear and direct to the reader, and provided just a single link to the donation page, with no button.

This led to a donation page that led with a headline that extends an offer to the reader: Invest in God’s work by strengthening a pastor’s marriage. This copy removed the story, with the hypothesis that the third-party story with no real connection and anonymized names might detract from the clarity of the value proposition. The treatment added a bulleted list of what the pastor couple would receive at a Weekend to Remember event, and made a clear ask to the reader. The visual separation from the green box was removed, and the donation form was shortened by bringing multiple fields (i.e. first name and last name) onto the same line, which gave the perception of less friction, while still collecting the same information.

They split their file evenly (making sure to evenly split donors and non-donors) and sent one version to each segment. The control email linked to the control page and the treatment email to the treatment page to preserve the integrity of the test throughout the entire appeal flow.

Research Question

How will a radical redesign of both the email and the landing page impact donor conversion?

Design

C: Control

T1: Radical Redesign

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence
C:	Control	0.01%
T1:	Radical Redesign	0.03%	272.1%	99.9%

This experiment has a required sample size of 27,274 in order to be valid. Since the experiment had a total sample size of 158,461, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

0% increase in traffic
× 272.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment donation flow produced a statistically valid 272.1% increase in donor conversion. This strong leap forward was due to a few identifiable conversion factors:

Reduction of visual friction in the control email led to more readers engaging with the copy.
Personal tone in the email increased motivation to give.
Decreased friction on the landing page increased likelihood to complete a donation

However, since all of these optimizations were made in one fell swoop, we cannot know exactly how each change contributed to the total increase. That said, this experiment provides a new benchmark for email appeal testing for FamilyLife—one that can be optimized in its own right.

Experiment Documented by Jeff Giddens

Jeff Giddens is President of NextAfter.

Question about experiment #6988

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.