Email Tests That Teach You Something
Email testing has collapsed into subject-line testing because platforms make that one click. It is also the variable with the smallest durable effect. An emoji that lifted opens by three percent on Tuesday does not generalise, does not compound, and does not tell you anything you can apply elsewhere.
Test in order of leverage
| Variable | Typical effect size | Transfers to other work? |
|---|---|---|
| Offer or proposition | Large | Yes — informs ads, landing pages, pricing |
| Audience or segment rule | Large | Yes — informs targeting everywhere |
| Send cadence | Medium | Partly |
| Body structure and call to action | Medium | Partly |
| Subject line wording | Small | Rarely |
Open rate has stopped being a measurement
Privacy features pre-fetch images, which inflates and distorts opens in ways that vary by mailbox provider. Treat opens as directional at best. The metric that survives is revenue or conversions per thousand recipients, which also has the advantage of penalising the clickbait that wins open-rate tests.
Optimise for the outcome you would defend in a meeting, not the one the dashboard puts at the top.
Do the arithmetic before the send
With a list of twenty thousand and a two-percent conversion rate, detecting a relative ten-percent improvement needs a sample most single campaigns cannot supply. Either test larger effects, accumulate across repeated sends of an automated flow, or accept that the result is a hint rather than a finding.
Write it down first
Before sending, record the hypothesis, the primary metric, the minimum effect worth acting on, and what you will do for each outcome. This single habit eliminates the most common failure in email testing, which is looking at six metrics afterwards and declaring victory on whichever moved.
Keep the archive
A test log with hypothesis, result and decision becomes, after a year, the most valuable marketing document you have. It stops the team re-running settled questions and gives new members the accumulated understanding of what this audience actually responds to.
Related articles
The Lifecycle Email Map Every Team Should Draw First
Campaign calendars are easy and mostly wasted. The revenue sits in a handful of triggered moments that most teams have never explicitly mapped.
Growing a Newsletter Without Buying Anyone
Fast list growth is easy to buy and impossible to keep. The compounding sources are slower and far more valuable.
List Hygiene: Sending Less to Earn More
A smaller list that reaches the inbox beats a large one that reaches the spam folder. It is not close.