Before you start
The path
1
1. Read the concepts once
Experiment IDs explains why every test needs a traceable identifier. Statistical significance explains what a result can and can’t tell you, including why checking early changes what the number means.
2
2. Fill in the experiment brief
Copy the experiment brief and fill in the hypothesis, the primary metric, the guardrail metrics, the sample needed, and the decision rule you’ll follow either way.Write the decision rule before launch. Deciding what counts as a win after seeing the data is how a flat test becomes a success story.
3
3. Register the ID
Issue an ID from your experiment ID pattern and record it before launch.
4
4. Apply the ID everywhere
The ID goes on every surface the test touches: campaign and ad names (conventions), UTM values, and the analytics events. This is what makes results traceable later.
5
5. Launch and leave it alone
Run to the sample size you calculated. Monitor the guardrail metrics for damage, not the primary metric for good news. See creating an experiment for the full process.
6
6. Review against the decision rule
Run the experiment review process. Compare the result to the rule you wrote in step 2, not to what you hoped for.
7
7. Record the learning
File the outcome in the experiment learning library, including the tests that showed nothing. A null result stops someone re-running the same test next quarter.
8
8. Roll it out or roll it back
A win becomes the new default and gets written into the relevant standard. A loss gets reverted and recorded. Either way the brief in the library carries the outcome.
The mistakes worth naming
Related resources
- Experiment brief The workbook.
- Creating an experiment The detailed process.
- Experiment review process How results get judged.
- Statistical significance What the numbers mean.
- Experiment learning library Where outcomes are recorded.
- Experimentation best practices How to run tests whose results survive scrutiny.