A/B Testing Cold Email: What to Test and What to Ignore

Everyone in outbound talks about A/B testing. Most do it wrong. They test subject line variations and call it optimization while ignoring the variables that actually determine whether a campaign books meetings or burns prospects.

What to Test (In Order of Impact)

1. Targeting

This is the single highest-impact variable and the one least likely to be A/B tested. Most teams pick an ICP once and never question it.

Test different audience segments head-to-head. VP of Sales versus VP of Marketing. Series A companies versus Series C companies. 50-person companies versus 200-person companies. The differences in reply rate between segments often dwarf every other variable combined.

2. Opening Line

The first sentence determines whether the rest of the email gets read. Test different opening approaches:

• Problem-first ("Most [title] at [stage] companies struggle with...")
• Observation-first ("Noticed that [company] just [trigger event]...")
• Question-first ("Curious — how is [company] handling [challenge]?")
• Direct ("I run AI outbound for companies like [company].")

3. Value Proposition Angle

Same product, different framing. Test whether prospects respond better to:

• Cost savings ("Save $100K per year versus hiring SDRs")
• Efficiency ("Book 3x more meetings with half the effort")
• Risk reduction ("Stop burning your domain on bad outbound")
• Competitive advantage ("Your competitors are already using AI outbound")

4. Call to Action

The CTA determines conversion from "interested" to "meeting booked."

• Soft CTA: "Would this be relevant to explore?"
• Direct CTA: "Do you have 15 minutes this Thursday?"
• Calendar link CTA: "Here is my calendar — grab a slot that works."
• Content CTA: "Want me to send over a case study?"

5. Sequence Length and Timing

How many emails and how far apart. Test:

• 3-email sequences versus 5-email sequences
• 2-day gaps versus 4-day gaps between emails
• Morning sends versus afternoon sends

What to Stop Testing

Subject Lines

The difference between a good subject line and a great subject line is 2-3% in open rates. The difference between good targeting and bad targeting is 200-300% in reply rates. Spend your time on targeting.

Fonts, Colors, and Formatting

Cold email should be plain text. Period. HTML formatting, colored buttons, and fancy signatures trigger spam filters and look like marketing emails. Stop testing design. Start testing message.

Signature Length

Nobody cares whether your signature has a phone number, three social links, and a company logo versus just your name and title. It does not affect reply rates. Move on.

How to Run a Valid Test

• Minimum sample size: 200 prospects per variation
• Single variable: Change one thing at a time
• Same audience: Split the same prospect list, do not compare different lists
• Sufficient time: Run for at least 2 weeks to account for delayed replies
• Primary metric: Reply rate, not open rate. Meetings booked, not replies.

Ted runs A/B tests automatically on Growth and Scale plans. Every campaign is continuously optimized based on real performance data, not gut feelings.