👋 Hey! Today is all about experimentation.
🧪 Experimentation is about testing decisions that involve uncertainty. In many cases, managers are uncertain about opportunities and they lack the data to inform strategic decisions. Through experimentation, you are able to slice out individual assumptions you have and validate them.
3 quick thoughts on how to introduce and scale experimentation
🥼 “Hi we’re Growth! Let’s experiment on everything! Sound good?”
It’s likely the growth team will be running more experiments than product or marketing teams. More language around ‘significance’ or ‘validation’ will start to pop-up over and over you might even get a fun eye-roll from your colleagues when dropping these words after a long zoom call. So how do you introduce & scale experimentation inside your organization? My quick thoughts:
Distributing the ownership of experiments to your team
💭 “I wonder what experiment is next?”
Typically a growth manager owns all experiments. At Jobber, we've distributed ownership to each team member. An experiment owner is accountable for the documentation, communication, execution, and analysis of an experiment. Here is how we did it:
Dealing with sample sizes (especially in B2B)
#️⃣ “That will take too long to run!”
^ me looking at our sample size calculator ^
Your target market isn’t every consumer in the world; It’s a subset of businesses looking to solve a particular pain point. As a result, the users you have to experiment with decrease substantially as you move further down the funnel. Do you give up?
Documenting your experiments with a proper template
🤔 “Didn’t we run that 6 months ago?”
Every experiment needs documentation. Our experiment template puts the proper pressure test on each idea and allows the team to point to something that is shareable and agreed on. Key stakeholders are informed before the document is shared publicly and the next steps tied to success or failure have buy-in from the group.
Here’s the template:
🤓 From Tal Reviv: Please, Please Don’t A/B Test That
👋 Thanks for reading!