The 'wine effect' and other surprises that reshaped how Box runs e-commerce experiments

In this episode of The Experimentation Edge, host Ashley Stirrup talks with Danielle Oleen, Director of E-commerce at Box, about what it really takes to build a culture of experimentation inside a B2B company. Drawing on 15 years across B2C and B2B at Wayfair, Drizly, Zoom, and now Box, Danielle explains why experimentation belongs to every product team and not just e-commerce, walks through a pricing page saga of one win and two losses that exposed the limits of simplification, and shares the "wine effect" test that won for a reason no one predicted. It's a practical, story rich conversation for product managers, growth leaders, and anyone trying to make better decisions with data.

00:45 Meet Danielle Oleen and Box's reinvention

02:45 Owning the entire customer life cycle

04:45 Why experimentation matters even without a checkout

07:45 The feature that's used but hidden

11:45 Proving ROI with a scrappy manual test

12:45 Building a culture that shares wins and losses

16:45 The pyramid strategy for prioritizing tests

18:45 The simplification tightrope on the pricing page

24:45 When a test wins for the wrong reason

27:45 Where experimentation at Box goes next

- Experimentation isn't only for e-commerce. Any product with a funnel, even an AI chatbot, can be measured and improved through testing.

- Simplification has a limit. Removing too much can strip away the cues and context buyers actually need to decide.

- Share losses as openly as wins. Wins build credibility, and losses build the psychological safety a testing culture runs on.

- Prioritize like a pyramid. Fix the widest-impact experiences first, then optimize down into smaller cohorts.

- Surprising results are the point. A test can win for a reason you never hypothesized, like the "wine effect," and that's where the real learning lives.

LinkedIn: https://www.linkedin.com/in/dolean1/

Website: https://www.box.com

Sponsor
GrowthBook is the warehouse-native platform for experimentation, feature flags, and product analytics trusted by AI-native product teams at 3,000+ companies worldwide.

Go to http://growthbook.io