A/B testing campaigns

A/B testing splits your audience between two variants of a campaign, measures opens or clicks on the test slice, then automatically rolls the winner out to the rest.

To run an A/B test:
1. Compose your base campaign as usual
2. Toggle "A/B test" in the Send panel
3. Configure the test:
• Variant A vs B subject (most common test) or Variant A vs B body content
• Test slice — % of audience used to test (default 20%; smaller slices need bigger audiences for significance)
• Winner metric — Open rate or Click rate
• Test window — how long to wait before picking a winner (e.g. 4 hours)
4. Click "Send test"

Sequence:
• At send time, the test slice is randomly split 50/50 between A and B
• Both variants go out simultaneously
• An hourly cron (sweepEmailABTestWinners) checks campaigns whose test window has elapsed
• The winner (variant with the higher metric) is sent to the remaining audience
• If the test is statistically inconclusive, the system defaults to Variant A

Where to see results:
• The campaign's Analytics dashboard shows variant A vs B performance side-by-side
• A "Winner: A" or "Winner: B" badge appears once rollout completes
• You can view per-variant opens, clicks, bounces, and unsubscribes

Best practices:
• Test one thing at a time — different subjects OR different body, not both
• Audience needs to be big enough that 50% of the test slice is ≥100 contacts, otherwise opens are too noisy
• Subject-line tests are usually higher-impact than body tests