Summary
What does experimentation look like inside a $150 billion retailer? In this episode of The Experimentation Edge, host Ashley Stirrup talks with Kim Ting Li, Senior Manager of Experimentation at The Home Depot, where one centralized team tests every major change to a $25 billion online business. Kim explains how 40 people serve 40–50 business teams, why executives join test readouts and ping analysts directly, how every result since 2020 lives in a searchable library, and why scaling beyond hundreds of experiments per year depends on server-side testing capabilities more than AI. For product, data, and engineering leaders building or scaling experimentation programs.
What does experimentation look like inside a $150 billion retailer? In this episode of The Experimentation Edge, host Ashley Stirrup talks with Kim Ting Li, Senior Manager of Experimentation at The Home Depot, where one centralized team tests every major change to a $25 billion online business. Kim explains how 40 people serve 40–50 business teams, why executives join test readouts and ping analysts directly, how every result since 2020 lives in a searchable library, and why scaling beyond hundreds of experiments per year depends on server-side testing capabilities more than AI. For product, data, and engineering leaders building or scaling experimentation programs.
Chapters
00:00 Intro
00:45 From neuroscience research to Home Depot
01:45 A $150B enterprise, a $25B online business
02:45 The centralized experimentation model
03:45 Inside the 40-person team
04:30 Readouts, blast emails, and the experiment library
05:40 Executive visibility and the golden rule
06:15 "If you won't act on a bad result, don't run the test"
11:15 Learning from losing tests
12:30 Scaling up: AI, server-side testing, and what's next
00:00 Intro
00:45 From neuroscience research to Home Depot
01:45 A $150B enterprise, a $25B online business
02:45 The centralized experimentation model
03:45 Inside the 40-person team
04:30 Readouts, blast emails, and the experiment library
05:40 Executive visibility and the golden rule
06:15 "If you won't act on a bad result, don't run the test"
11:15 Learning from losing tests
12:30 Scaling up: AI, server-side testing, and what's next
Takeaways
- One centralized team of about 40 people tests every major change to Home Depot's $25B online business, serving 40–50 business teams with consistent hypothesis and analysis standards.
- Executive engagement is real at Home Depot: leaders join 30-minute readouts, search the experiment library, and ping analysts directly because they treat A/B testing as the golden rule for measuring incrementality.
- Institutional memory is infrastructure — every test result since 2020 lives in a centralized, searchable archive so no one re-runs a question the company already answered.
- Kim's stakeholder filter: if you wouldn't do anything differently after a bad result, don't run the test.
- Scaling past low hundreds of experiments per year is a capabilities problem before it's an AI problem — Home Depot is moving from client-side to server-side testing so winners release quickly, end to end.
Connect with the Guest
LinkedIn: https://www.linkedin.com/in/kimtingli
Website: https://www.homedepot.com
LinkedIn: https://www.linkedin.com/in/kimtingli
Website: https://www.homedepot.com
Sponsor
Growthbook helps you ship features with confidence by bringing experimentation and feature flagging into one open-source platform. No more guessing whether that new checkout flow actually moved the needle, waiting weeks for data team bandwidth, or flying blind on rollouts.
Growthbook gives you a single place to run A/B tests, manage feature flags, and analyze results against your existing data warehouse.
With powerful stats built in, it takes the complexity out of experimentation, helps you catch regressions before they hit every user, and makes it easy to test ideas that keep your product improving and your metrics moving in the right direction.
See a demo at https://www.growthbook.io/