Every experiment needs documentation. Our experiment template puts the proper pressure test on each idea and allows the team to point to something that is shareable and agreed on. Key stakeholders are informed before the document is shared publicly and the next steps tied to success or failure have buy-in from the group.
Here’s the template:
Hypothesis
Statement about what you believe to be true today. Not what you think is going to happen, but what you believe is true about the idea itself. Starts with "we believe" It probably has the word 'because' It doesn't contain "if" or "then" (that's a prediction)
Background & Strategic Fit
Why is it important that we learn the above? Why is it a priority?
Assumptions
List any assumptions that we are making for this experiment, often resulting from outcomes of the questions above. Generally, your experiment is looking to validate one of these assumptions.
Prediction
What impact do you think this idea will have? What is the metric we are looking to improve? What is the lift we are predicting?
Historical Data
What have we seen in the past? What is the baseline data we can gather? Show the opportunity
Next Steps
If the experiment succeeds, we will ________
If the experiment fails, we will _______
Design
If we have a prototype or visual, this is the place to put it. Ideally, record a video.
Sample Size Calculator
What sample size will you need? How long will it take to run? Add a screenshot of your sample size calculator.
Not doing
List requirements/functionality that are deemed out of scope. Set expectations with stakeholders, and focus the scope of the team
Analyzing Results
Success or Failure?
Determine if the experiment was a success or a failure. Did the experiment improve the metric in the prediction?
Apply Learning
Based on what we learned, what actions can be made moving forward?
Secondary Results/Observations
Outside of the key results and learning, are their other metrics or observations worth noting as it relates to this experiment?
What does your documentation look like?