• Agile & Scrum Basics

How to get the most out of prototypes and experiments? The experiment canvas!

We innovate three beats around the bush. Just when it looks like your previous innovation is going to run, comes the next question: if we try it this way once? Or this way? Business contributes one idea after another to development, but there’s no time to evaluate what anything delivers. Maybe you use Scrum for your innovation and your Product Backlog is full of experiments. Before you know it, you’ve done experiment upon experiment and you start doubting what conclusions you can still draw. This article gives you a handy tool to give structure to your experiments: the experiment canvas. You can download and print it or use it with your team in your online whiteboard.

Assumptions are the basis for a good experiment

All the ingenuity of our modern technology is possible thanks to the scientific method, which has been around since the 17th century. The scientific method is an empirical method for acquiring knowledge. The method requires careful observation. To observe well, the basic premise is that you must be very suspicious of your own assumptions. In fact, those assumptions can greatly distort the interpretation of the observation!

But then how exactly are those assumptions the basis for the experiment? You can think of assumptions as theory and experiments as practice. With a good experiment, you find out whether your theory is likely to work in practice. If you formulate an assumption in such a way that you can test it, it’s called a hypothesis. That’s a hard word for a proposition that you can disprove with an experiment. Some examples:

• There is a market for luxury, 100% electric cars
• Dust masks help reduce chance of infection
• During lockdown, your Scrum Academy clients are better served with on-the-job guidance than with training.
• My mother would like an app with recipes
• Mass m and energy E are two different names are for the same underlying quantity: E = mc²

Einstein had come up with that formula on a hike. It seemed to go very well, but until someone proved it with a particle accelerator, it was a hypothesis. So in the experiment canvas, the hypothesis is central. But, of course, a good experiment requires more than an interesting assumption. The design of the experiment and expectations also play a role.

How you measure something determines what conclusions you may draw

Take the example of mouth caps. There is much debate about whether they actually help or not. It matters quite a bit whether you touch them with your hands, how long you use them and what kind of material they are made of. So if you want to discover whether mouth guards help to transmit less disease, you need to properly record how you proceed when measuring.

The experiment will look very different if you approach the mouthguard from a design or regulatory perspective. The designer asks what should mouthguards look like so that people use them safely and often? The policy maker asks what rules are enforceable and where there is enough support. Whatever the experiment looks like, it is important to think about it carefully beforehand. Will your approach allow you to draw the conclusions you want?

Designing experiments is a creative activity. There is an art to measuring exactly what you want to know in a very elegant way. Design Thinking can also help you design the experiment, not just the design of your final product or service. The experiment canvas is therefore covered extensively in the Design Thinking & Lean Startup training.

Predict the outcome to get rid of your own bias

You’ve formulated a hypothesis and come up with a test design. Before you get started, it’s important to take a guess! What do you expect the result to be? A guess is not only exciting, the guess also helps counteract hindsight thinking. When you look back on an experiment, the result makes sense. But did you expect the same beforehand?

It is also true that many hypotheses have emotional overtones. Maybe you’re just a big opponent of mouth caps. Before you know it, you’re going to use every result of the mouthguard experiment to argue that they don’t help. For designers, this can also be complicated. If you’ve worked on a design for a long time, you want it to do to people what you expected. You then seize on every test result to substantiate that your design was good.

So to reduce hindsight thinking, it is smart to determine in advance at which measurement results you want to start drawing which conclusions. The experiment canvas challenges you to set the boundary in advance of the experiment when you decide whether your assumption is correct.

Conclusions are best drawn as a team

When the results of the experiment are in, reality is usually not clip and ready. The test went differently than you thought, or the results are unexpected. You may also have progressive insight about your approach. Did you proceed in the right way?

If you’ve followed the approach in this article, that’s no problem at all. You’ve thought through the hypothesis well, and clearly described your test design. The results are in and you can analyze with your team what happened. The better you’ve captured everything, the more valuable the analysis. More data means firmer conclusions. And much more importantly, by capturing the experiment well, you can involve more people and thus gather more perspectives on what happened. When you Scrump, you do this with your team and your stakeholders. In science, this is called peer review.

Even after the experiment, the experiment canvas clearly shows what was intended and how the experiment went. It fits on 1 A4 sheet and facilitates the conversation in your team.

Conclusion: A good experiment is a documented experiment

When you work Agile you are working on value all the time. A well-documented experiment is more valuable than a quick trial balloon:

• You figured out in advance what you want to get out of it
• You can run it with several people and add up the results
• You can look afterwards to see if your assumptions were correct
• Other people can reproduce your experiment to see under what conditions the conclusions remain true
• You can involve more people in the analysis of results and draw better conclusions

Exactly to make sure you don’t overlook anything when designing experiments, we developed the experiment canvas. You can download it as a pdf and print it out a few times or you can use it in your favorite online whiteboard. In any case: fill as many canvases with experiments as possible and do it with your team. That way you’ll be sure to do the most valuable experiment first.

Experiment Canvas Download?

Leave your name and email address and we will immediately send you a download link for the tool to better structure your experiments!

Photo by fauxels from Pexels