ProductiveInnovation
In today’s digital world, more and morecompanies are turning to experiments to discover ways to create or improveonline experiences. In this issue HBR looks at what it takes to develop thecapacity
to do large-scale testing and use it tolift firm performance. | page 053
Buildinga Culture of Experimentation
Stefan Thomke | page 054
Online experiments can be a game changerwhen it comes to marketing and innovation. By running some 25,000 tests a year,for instance, Booking.com has transformed itself from a small start-up to theworld’s largest accommodation platform. Today scaling up an organization’sexperimentation capabilities is critical, but many firms struggle to do it—notbecause of technology but because of culture.
To break down cultural barriers, companiesneed to create an environment where curiosity is nurtured, data trumpsopinions, any employee can launch tests, all experiments are ethical, and a newmore-democratic model of leadership prevails. Ultimately, executives have to beable to confront the possibility that they are wrong daily and willing to givetheir people greater autonomy.
Avoidthe Pitfalls of A/B Testing
Iavor Bojinov, Guillaume Saint-Jacques, andMartin Tingley | page 062
Online experiments measuring whether “A,”usually the current approach, is inferior to “B,” a proposed improvement, havebecome integral to the product-development cycle, especially at digitalenterprises. But often firms make serious mistakes in conducting these tests:They focus on the average, instead of looking at how a change impacts differentcustomer segments. They forget that customers are connected and that theirinteractions can affect test outcomes. And they run tests for too short aperiod, failing to recognize that customers’ reactions can change over time.This article describes how to avoid all those traps by applying techniques thatLinkedIn and Netflix have used to produce better insights.
“The Power ofThese Techniques Is Only Getting Stronger”
A conversation with Pinterest’s Jeremy King| page 068
Jeremy King has spent much of the pastthree decades helping firms such as eBay, Walmart, and his current employer,Pinterest, use experimentation and data to improve decision-making. In thisarticle he shares what he’s learned about the need to balance precision withserendipity, how to promote “data democracy,” and the importance of investingin the right training for employees.