A/B Test Planning Document
Create a comprehensive A/B test plan for [feature/element]. Include: 1) Clear hypothesis statement in the format 'If we [make this change], then [this outcome] will happen, because [rationale]', 2) Primary and secondary success metrics with specific calculations, 3) Required sample size based on expected effect size, 4) Test duration calculation based on traffic volume, 5) List of potential confounding variables and mitigation strategies, 6) Test implementation requirements, and 7) Analysis plan including segmentation dimensions.
How to Use This Prompt
- Copy the prompt using the button above
- Replace placeholders in [brackets] with your specific details
- Paste into your AI assistant (ChatGPT, Claude, Gemini, etc.)
- Iterate as needed - ask follow-up questions to refine the output
Why This Prompt Works
This prompt forces the AI to structure your A/B test around a rigorous hypothesis format and tie it to specific, calculable metrics. By requiring sample size calculations and confounding variable analysis upfront, it prevents the common mistake of running underpowered tests or drawing conclusions from noisy data. The output gives you a document you can share directly with data science and engineering teams.
When to Use This Prompt
- Before launching a new feature variant and you need to define what success looks like with statistical rigor
- When stakeholders are debating two design approaches and you need a structured experiment to settle it with data
- During quarterly planning when you want to build an experimentation roadmap with proper test sequencing
Tips for Better Results
- Include your current conversion rate or baseline metric so the AI can estimate realistic sample sizes and test durations
- Specify your daily active users or traffic volume so duration calculations reflect your actual scale
- Mention any upcoming events (holidays, launches) that could act as confounding variables for more accurate mitigation plans