The replication crisis of unreliable A/B test winners

9,1 on Springest

A/B testing remains a highly reliable method for validating assumptions. However… the truth is that the current practice of A/B testing is experiencing a replication crisis. Which means that if you were to conduct the same test again, would you get the same results? A significant portion of your winners are not winners at all (false positives), and a significant portion of your losers are actually winners (false negatives). Is your business case still as strong as you think? And yet, this can be prevented.

In this special masterclass, Bart Schutz and Kobe Millet will illustrate what the ‘Replication crisis of A/B testing’ entails and teach you how to multiply the impact of A/B testing in your organization. It is a masterclass for any team and organization involved in or about to ingage in A/B testing and is willing to confront the beast head-on.

Sign up for the masterclass
bart

Course content

In this Masterclass, you and your team(s) will learn:

  • What the replication crisis is. The field of A/B testing is facing a replication crisis. Many winning A/B tests are false, and most effects are exaggerated. Moreover, most companies fail to build reliable customer knowledge based on their A/B tests. You will learn the truth.
  • The human factors causing this crisis and how to prevent them.
  • The methodological factors causing this crisis and how to address them.
  • How these recent insights lead to a multiplication of the impact of A/B testing on both the commercial goals (earn) and customer intelligence goals (learn) of your A/B tests.

What will you learn?

  • Insights directly from the A/B testing masters.
  • A better understanding of the ‘perverse stimuli’ in testing processes.
  • Clear visibility into the share of false A/B test results.
  • Guidance on building a more reliable experimentation process and culture.
  • Toolset featuring methods and metrics to map out ’the truth’.
  • An Earn & Learn framework with more reliable learnings from your A/B tests, and an amplification of the actual contribution to revenue growth.

Additional information:

Are you already looking forward to this in-depth in-company Masterclass? Reserve your seat now!

  • Format: Two afternoons of inspiration and training (with one week in between for homework)
  • Location & Time: In-company (can be discussed, potentially in Utrecht)
  • 1:00 PM – 5:00 PM (both sessions)
  • Price: €7,500 (excluding VAT)
  • Certificate: Yes (proof of successful participation)

The replication crisis of unreliable A/B test winners

Interested in this Masterclass? Sign up!