When do I typically need A/B tests?

Here are just a few use cases you may experience in your day to day work.

Redesign your most important landing page

One of your most important landing page has been redesigned to better showcase the value of your services, and you need to make sure the new version is not making things worse. To compare the performance of two new versions and check the new version does not decrease number of contact form signups or the , you run a A/B test and see whether the redesigned version is converting more visitors to reach your goals.

Optimise your conversion funnel

As a marketer or product manager, you often need to improve conversions at any particular step of the conversion funnel. Basically you need more visitors to get closer to the end of the funnel and convert your goals (whether signing up, leaving an email address, downloading a brochure, watching a video, etc.). To improve your conversions you may start by identifying the bottlenecks and what is causing users to leave the funnel and where. For example, you may be losing many users at a particular step of your conversion journey such as your signup form or your checkout page. You and your team may then brainstorm several solutions that would make things better, and your team decides to implement the most important change first. When you implement this change (and any others) using A/B Testing, you are actually now scientifically measuring and the A/B testing product will confirm (or deny) that your change has indeed improved conversions (and how much improvement).

Soft launch and test your new feature (advanced)

As a product or game designer, you are building an app, game or any other software product, and you wish to soft launch a new exciting and innovative feature. Soft Launching is a great way to get ideas tested early without much risk. Soft launching means that your new feature will be visible only on a subset of users and compare behaviour. So you can use A/B Testing to soft launch your apps and games features, and measure how the new feature performs on for example 10% of your user base. A/B Testing indicates whether users are better engaged in the app (across all your success metrics), after using the new feature.

See more example in the blog post: A/B Testing Platform for Matomo Analytics (formerly Piwik Analytics).