Before launch
At launch
After the test ends
Setup
Fill in the fields that identify the test and who owns it. Keep the rows that fit, add any your team relies on:Concept review: Setup fields
Concept review: Setup fields
Test type
E.g A/B: one control, one variantConcept review: Why test type is worth naming
Concept review: Why test type is worth naming
Hypothesis and metrics
Hypothesis
State the change you’re making, the outcome you expect, and why: E.g. Changing the pricing-page CTA button from gray to orange will increase click-through rate, because orange has higher contrast against the page background and draws the eye more clearly.Concept review: What makes a hypothesis useful
Concept review: What makes a hypothesis useful
Primary metric
Name the single metric you’ll use to decide if the test worked: E.g. Click-through rate on the pricing-page CTA.Concept review: The role of a single primary metric
Concept review: The role of a single primary metric
Secondary metrics
List metrics you’ll watch for the fuller story, but won’t use to make the decision: E.g. Time on page, scroll depth, and downstream checkout conversions.Concept review: What secondary metrics add
Concept review: What secondary metrics add
Guardrail metrics
List the metrics that must not degrade, even if the primary metric improves. Common ones are below; keep, cut, and extend: E.g. For the CTA test, Doughnut Labs watches revenue per session and average order value; if either drops more than 2%, the test doesn’t count as a win.Concept review: How guardrails protect a result
Concept review: How guardrails protect a result
Baseline
Record the current performance of the primary metric before the test runs: E.g. Current pricing-page CTA click-through rate is 4.1% over the trailing 30 days.Concept review: Why the baseline is captured first
Concept review: Why the baseline is captured first
Test design
Success criteria
State what you’ll accept as a win, before results come in. A common shape combines a lift, a confidence threshold, and a guardrail condition: E.g. At least a 5% relative lift in CTA click-through, at 95% confidence, with no more than a 2% drop in average order value.Concept review: Deciding the win before you can see it
Concept review: Deciding the win before you can see it
Sample size and minimum detectable effect
Record the sample size the test needs and the smallest effect it can reliably detect at that size: E.g. 18,000 sessions per variant, giving a minimum detectable effect of roughly 4% relative lift. Calculated before launch with the sample size calculator.Concept review: Sizing the test before you run it
Concept review: Sizing the test before you run it
Statistical significance
Set the confidence threshold you’ll use, and note what you’ll do if a test can’t reach it: E.g. 95% confidence (p < 0.05). The pricing page has enough traffic to reach this. For lower-traffic tests, Doughnut Labs records the observed direction, notes that significance wasn’t reached, and makes a directional call.Concept review: Significance, and when you can't get it
Concept review: Significance, and when you can't get it
Test timeline
Record the planned start and end dates, derived from the sample size math: E.g. Start 2026-07-14, end 2026-07-28. End date set by the sample math (18,000 sessions per variant at current traffic), not by the calendar.Concept review: Why the end date comes from the math
Concept review: Why the end date comes from the math
Test description
Record the test in enough detail that someone auditing it in six months could find the exact assets and settings.Concept review: Writing for the six-month auditor
Concept review: Writing for the six-month auditor
Variants and splits
List every variant, including the control, and how traffic, budget, or customer profile divides between them. Splits should total 100%. Add a row per variant for A/B/n or multivariate tests:On-site tests
A page variant has no name in any of the naming conventions, because those cover ad platform objects and a testing tool owns its own. What fills the platform object name column above is the tool’s experiment and variation name, plus the ID.Concept review: Naming an on-site test
Concept review: Naming an on-site test
Concept review: Listing the control and the splits
Concept review: Listing the control and the splits
Results
After the test ends, mark it with a status and note the outcome. These five cover the common cases; add your own if your team distinguishes more:Concept review: Statuses and the link to a report
Concept review: Statuses and the link to a report
Related resources
- Experiment brief concepts
- Experiment ID conventions and the experiment ID concepts
- Statistical significance
- Experiment review process
- Experiment learning library
MCP wiring
MCP wiring
Domains: experiment, strategy
Provides:
experiment-briefUses:
campaign-brief, copy-brief, creative-brief, experiment-id, experiment-log, glossary, library:site-page, naming-convention, process:experiment-review, statistical-significanceTriggers: “Help me set up an A/B test” · “Write an experiment brief” · “I want to test a new headline against the current one” · “Plan a creative test on Meta” · “How big a sample do I need for this test?”Asks, in order:- What are you changing, and what do you expect to happen because of it? Options: your answer.
- What kind of test is it? Options: experiment-brief#test-type. Skip when: only one arm vs control implied (default A/B).
- Which single metric decides the win? Options:
kpi-definitions. - Which metrics must not get worse? Options: experiment-brief#guardrail-metrics.
- What’s the primary metric’s current baseline? Options: your answer. Skip when: flat file accessible.
- What lift and confidence count as a win? Options:
statistical-significance. Skip when: team default set. - What’s the control’s exact platform name? Options: your answer. Skip when: control built fresh.
- Is this an on-site test? If so, which tool and same or separate URL? Options: your answer. Skip when: ad platform test.