Skip to main content
Use this whenever you’re changing something to find out whether it works, rather than because you already know it does.
This path assumes a control to compare against. If you’re spending to find out whether a new channel, product line, or market is worth continuing, there is nothing to compare it to and this process doesn’t fit. Use the pilot brief instead: it sets a read date and a kill or scale rule in place of a control, and its result files into the same learning library.

Before you start

The path

1

1. Read the concepts once

Experiment IDs explains why every test needs a traceable identifier. Statistical significance explains what a result can and can’t tell you, including why checking early changes what the number means.
2

2. Fill in the experiment brief

Copy the experiment brief and fill in the hypothesis, the primary metric, the guardrail metrics, the sample needed, and the decision rule you’ll follow either way.Write the decision rule before launch. Deciding what counts as a win after seeing the data is how a flat test becomes a success story.
3

3. Register the ID

Issue an ID from your experiment ID pattern and record it before launch.
4

4. Apply the ID everywhere

The ID goes on every surface the test touches: campaign and ad names (conventions), UTM values, and the analytics events. This is what makes results traceable later.
5

5. Launch and leave it alone

Run to the sample size you calculated. Monitor the guardrail metrics for damage, not the primary metric for good news. See creating an experiment for the full process.
6

6. Review against the decision rule

Run the experiment review process. Compare the result to the rule you wrote in step 2, not to what you hoped for.
7

7. Record the learning

File the outcome in the experiment learning library, including the tests that showed nothing. A null result stops someone re-running the same test next quarter.
8

8. Roll it out or roll it back

A win becomes the new default and gets written into the relevant standard. A loss gets reverted and recorded. Either way the brief in the library carries the outcome.

The mistakes worth naming

Type: process
Domains: experiment
Provides: task:run-an-experiment
Uses: data-dictionary, experiment-brief, experiment-id, experiment-log, naming-convention, pilot-brief, statistical-significance, utm-rules
Hands off to: process:experiment, process:experiment-review
Triggers: “I want to test something” · “Run an A/B test” · “Is this worth testing?” · “Try a new channel”Asks, in order:
  1. Is there a control to compare against? Options: yes or no. Skip when: request names control.
  2. What’s the hypothesis and decision rule? Options: your answer. Skip when: never skip.
Output: Routes to creating-an-experiment (control) or pilot brief (no control); result filed in learning library and written into the standard.

Get the complete Marketing OS

Subscribe to Snow Doughnut, our free weekly newsletter, and get the Notion workspace where you do the work: tasks, a marketing calendar, name generators, briefs, and more.

Have us run it for you

We implement this system, use it to help you grow and scale, then hand it all off to you when you’re ready.
Last modified on November 2, 2026