- Marketing Herald
- Posts
- Most A/B tests are a waste of time.
Good Morning
Table of Contents
Marketing News
Most A/B tests are a waste of time.
Brands burn 1,000s of hours every year testing tiny tweaks that don't have an impact.
And usually, it comes down to skipping one important question:
“Why am I even testing this?”
If you can’t clearly explain what you’ll do differently depending on the result - you probably shouldn’t run the test.
I’ve run hundreds of tests across email & SMS.
Here’s what I've learned about strategic testing:
1) Only test if there’s enough traffic.
↳ Testing low-traffic flows or branches which get 10 deliveries a week? You’re wasting months for a tiny lift that won't mean anything.
→ Focus on high-volume touchpoints first: welcome flow, abandoned cart, opt-in forms.
→ Smaller brands? Leverage qualitative data, or use high-traffic campaigns as a testing ground. Don’t waste your time on tests that won’t hit statistical significance.
2) Tiny tweaks don’t deliver big results.
↳ Testing button colours, emojis in subject lines, or tiny wording changes in your footer?
Unless you're Google, you’re wasting your time. The results are rarely meaningful enough to justify the effort.
→ Instead, test BIG ideas.
For example: Mystery offers are one of my favourite tests for email opt-ins. Opt-in rates might not always jump - but welcome offer utilisation? (the people actually redeeming the offer) usually does.
Why? Psychology. Mystery triggers curiosity, making people feel like they’ve earned something special, so they’re far more likely to actually spend it.
3) Statistical significance matters - but don’t wait forever.
↳ Ideally, wait for significance. But if your test is clearly tanking vs the control, stop early.
→ Don’t spend weeks proving a bad idea wrong when you could be testing 2 or 3 other variations instead.
4) Prioritise the highest-impact tests first.
↳ When auditing a new brand, I usually start with the welcome offer.
→ Reducing your welcome offer from 15% to 10% without reducing conversions? Instant profit gains.
→ Yet many brands stick with a 10% discount without ever testing cashback, fixed amounts ($ off), or smaller incentives.
If you’re going to test, test strategically:
→ Big-impact ideas first
→ High-traffic touchpoints
→ Bottom-line impact over vanity metrics
→ Make a change based on the result.
Do that, and you’ll stop burning hours on meaningless lifts - and start running tests that genuinely change your business for the better.
Most people don’t click Meta ads. And that’s exactly the point.
They’re not ready to buy.
But they remember you.
Then, a few hours (or days) later…
→ They Google your brand
→ Click your site
→ Buy something
That’s how demand gets created.
Quietly. In the background.
But here’s the mistake:
You paid to create demand…
…and your competitor shows up first on Google.
They close the sale you started.
So here’s the move:
1. Run top-of-funnel Meta to create demand
2. Run branded Search to catch it when it shows up
Discovery + intent.
Attention + conversion.
Brand + brain.
If you’re not connecting the two, you’re lighting money on fire.
Most email flows are built for marketers, not customers.
Here’s what I mean:
Your flow says:
Day 1 - Welcome
Day 3 - Story
Day 5 - Offer
Day 7 - Reminder
Day 10 - Retarget
But your customer’s journey is not a calendar. It’s a context engine.
They might be:
• Ready to buy instantly
• Still comparing 3 brands
• Trying to figure out if your product even fits their life
So why are we sending the same 7 day flow to everyone?
Instead of a linear flow, think in states:
→ “Has clicked but not purchased”
→ “Landed on FAQ page”
→ “Browsed 2+ times in 48 hrs”
→ “Said they care about sustainability in quiz”
Flows should evolve based on behavior.
Because inboxes aren’t empty rooms.
They’re battlegrounds for attention.
Everyone talks about VSL Funnels…
But unless you have strong follow-up processes, your funnel is toast.
Here’s how I structure high-converting VSL follow-up sequences (using ActiveCampaign):
Day 1 (Immediate):
Deliver the VSL replay link via email and SMS
Reinforce the biggest takeaway and reframe the core problem
Day 2:
Email: “Did you see [Powerful Insight from VSL]?”
Summarize key points (bullet format)
Introduce a compelling client result or short case study
Day 3:
Email: Objection handling
Tackle common objections directly, provide reassurance, and reinforce credibility with proof or testimonials
Day 4:
Email: Scarcity & urgency
Offer a limited-time bonus or incentive to act now
Highlight a specific client transformation
Day 5:
Final call (Email + SMS reminder)
Gentle “last chance” CTA: "If you’re serious about solving [core problem], let’s connect now."
Why ActiveCampaign?
✅ Automations make it easy—build sequences once, and let the system do the rest
✅ Advanced segmentation helps you speak directly to different audience segments
✅ Easy tracking lets you see exactly who’s engaging, who’s ready to buy, and who needs more nurturing
If you’re running a VSL without a structured follow-up, you’re missing out on at least half your sales.