Sometimes, the smallest tweaks can lead to the biggest gains. In this experiment, we tested a subtle popup change — swapping the traditional “x” close button with a gentle “Maybe Later” prompt — across multiple clients. A simple adjustment led to higher engagement and increased traffic to the donation page. Here’s a closer look at what we found and why this low-friction nudge might be worth testing in your next campaign.
Research Question (Hypothesis):
If we introduce a small amount of friction on the popup by replacing the simple “x” icon with a button requiring the user to actively reject giving help (using gentle language like “Maybe Later”), we will see a higher gift count. This is based on the idea that our ideal prospect is a caring individual who may hesitate to outright refuse to help others.
Summary:
The theory is that this change appeals to people who genuinely care about the cause but are undecided about donating. Since the decision to give is rarely instantaneous — and is influenced by many variables — this test added one more subtle nudge in the direction of generosity.
Set Up and Time Frame:
This experiment was run from October 28 to November 5, 2024. We A/B tested the control and variant popup versions with all incoming site traffic.
Results:
Key Learnings and Recommendations:
- The variant achieved a 35% higher click rate than the control at a 95% confidence level.
- Our secondary KPI — donation completion rate — also favored the variant, which saw a 50.69% higher giving rate than the control during the test. While this result wasn’t statistically significant, it strongly suggests the variant is more effective.
- This experiment successfully increased traffic to the donation page.
- We tested this approach across several clients in a similar time frame and saw the same pattern emerge. While results may vary depending on your audience, this subtle prompt consistently improved click-through rates.
Let us know if you try out this test, we want to hear what you think!
