> ## Knowledge Base Index
> Fetch the complete knowledge base index at: https://support.wiraya.com/sitemap.xml
> Use this file to discover available pages before exploring further.
> Pure-Markdown content can be obtained by appending a '.md' suffix to the content URLs listed in the sitemap (without the trailing slash).

# A/B Testing Checklist: Before You Call a Winner

# A/B Testing Checklist: Before You Call a Winner
###### Use this checklist to ensure your A/B test results are reliable before launching or declaring a winner.

It’s easy to jump to conclusions when early results look promising—but poorly designed tests can lead to misleading outcomes. This checklist helps you validate that your test setup, measurement, and interpretation are sound.

## **1. Setting Up the Test**
Before launching, make sure your test is structured correctly:
* Test **only one variable at a time**
* Agree on **sample size upfront** (recommended: at least 1000 per group)
* Define **test duration in advance** — avoid stopping early
* Ensure both **A and B groups run simultaneously**

## **2. Measuring the Result**
Make sure your measurement approach is solid:
* Define **one primary metric** before the test starts
* Confirm **outcome data** (e.g. paid / not paid, payment date) with the customer
* Avoid **overlapping tests** on the same audience
* Ensure **external factors** (e.g. holidays, month-end spikes) do not skew results
* Collect **enough data** before deciding (not just a handful of responses)

## **3. Reading the Results**
Interpret results carefully and objectively:
* Don’t rely only on averages — **segment results** (e.g. age, gender)
* Check both **reach rate AND conversion (e.g. payment rate)**
* **Document results**, even if the test fails
* Don’t change the **success metric after seeing the data**
* **Share learnings** with the wider team

## **Summary**
Before calling a winner, ensure:
* The test was set up correctly
* The data is reliable
* The results are interpreted objectively
Following this checklist helps you avoid false conclusions and build a more data-driven approach to optimization.

###### Read more
[Getting Started with A/B Testing in Optimized.App](https://support.wiraya.com/en/article/getting-started-with-ab-testing-in-optimizedapp-jfdzbp/)
[Payment Reminder Content A/B Tests: What You Say & How You Say It](https://support.wiraya.com/en/article/payment-reminder-content-ab-tests-what-you-say-how-you-say-it-1p0kgb6/)
[How to use A/B/C Testing Action in Sequence Campaign?](https://support.wiraya.com/en/article/how-to-use-abc-testing-action-in-sequence-campaign-1m0xocf/)