High Signal: Data Science | Career | AI

Martin Tingley, Head of Windows Experimentation at Microsoft and former Head of the Experimentation Platform Analysis Team at Netflix, talks about why humans are the bottleneck in experimentation, and how a five-level maturity framework points the way toward self-optimizing software.

Our conversation traces the path from basic hypothesis testing to a frontier where Generative AI creates, evaluates, and refines product variants in a closed loop. We explore the architectural shift required to move from testing single variants to optimizing entire parameter spaces, and how startups are already using AI to generate production-ready landing pages for Fortune 500 companies in hours rather than weeks. Tingley also shares a strategic lens on "experimentation programs," explaining how plotting the distribution of treatment effects across different product areas can serve as a powerful tool for capital allocation and high-level strategy.

LINKS

Martin on LinkedIn

Want Your Company to Get Better at Experimentation? by Iavor Bojinov, David Holtz, Ramesh Johari, Sven Schmit and Martin Tingley (Harvard Business Review)

Avoid the Pitfalls of A/B Testing by Iavor Bojinov, Guillaume Saint-Jacques and Martin Tingley (Harvard Business Review)

Martin & Co.'s Seven Part Blog Series on Experimentation at Netflix

Roberto Medri (Meta) on High Signal: The Incentive Problem in Shipping AI Products — and How to Change It

Tim O’Reilly on High Signal: The End of Programming As We Know It

Watch the podcast episode on YouTube

Delphina's Newsletter

Bozena Pajak, VP of Learning at Duolingo, joins High Signal to discuss the evolution of AI at Duolingo: from personalized difficulty models to the current generative frontier where AI characters provide low-stakes and high impact conversational practice. We discuss the role of AI in overcoming one of the biggest hurdles in language acquisition, speaking anxiety. We also talk about how Bozena's team leverages agentic workflows to scale content and why the next wave of personalization involves shifting from difficulty levels to "thematic lenses" tailored to specific user interests.

LINKS

Bozena on LinkedIn

The original AI: how your brain tracks language patterns, a Duolingo blog post

How Duolingo uses AI to create lessons faster, a Duolingo blog post

Duolingo is hiring a Learning Scientist (Efficacy Research), a Director of Learning Design (Language Learning), and a Director of Learning Design (Immersive Language Learning)

High Signal podcast

Watch the podcast episode on YouTube

Delphina's Newsletter

Benn Stancil, writer and co-founder of Mode, joins High Signal to ask some uncomfortable questions about the current AI moment. Is now actually a terrible time to start a company? If the tools you build on today are obsolete in six months, at what point does the head start stop mattering? Is all that context engineering you're doing a waste of time, destined to go the way of Boolean search syntax in the 90s?

Benn argues that AI is turning us all into Steve Jobs, not the visionary who delegated, but the one who berated people over pixel placement. As AI takes over the doing, our job becomes obsessing over the polish. He makes the case that technical debt may be self-healing: if future models can untangle the mess today's models made, then messy code isn't debt…it's a spec for a clean rewrite.

We also dig into why Claude Cowork can't work. AI has these uncanny ticks you can't beat out, so anything it writes "as you" will smell like AI. The solution isn't better AI writing—it's to stop pretending we write to each other at all. Benn envisions a future where communication is radically intermediated: I dump facts into a shared repository, your AI reads them, and nobody bothers with the social decoration in between.

LINKS

Benn’s blog on Substack

Benn.website, with links to all everything else Benn related

Will there ever be a worse time to start a startup? Today's frontier is tomorrow's tech debt.

Why Cowork can’t work: The future isn’t collaborative.

Producer theory: Platforms are overrated.

Tim O’Reilly on High Signal: The End of Programming As We Know It

Watch the podcast episode on YouTube

Delphina's Newsletter

Nicholas Moy, former Head of Research at Windsurf & now at Google DeepMind, joins High Signal to discuss the shift from "co-driving" to a truly "agentic" era of development. We discuss Windsurf's journey from early prototypes that struggled with compounding errors to the successful launch of their agentic coding product. Nick explains that building a startup in the current climate requires a strategy of "disrupting yourself" to avoid the innovator’s dilemma; companies must be ready to pivot as soon as a new frontier model makes previously impossible features viable. He argues that traditional technical moats are increasingly fragile, and true defensibility now comes from real-world usage data, brand reputation, and a deep intuition for what users need at the frontier of these capabilities.

LINKS

Nicholas Moy on LinkedIn

Introducing Google Antigravity, a New Era in AI-Assisted Software Development

“A Flash of Deflation - Gemini 3 Flash represents a step function increase in model deflation : a gauntlet thrown” by Thomas Tunguz

Tomasz Tunguz on Why a Trillion Dollars of Market Cap Is Up for Grabs (and How AI Teams Will Win It)

High Signal podcast

Watch the podcast episode on YouTube

Delphina's Newsletter

Cara Dailey, VP and Head of Data Strategy at Early Warning (the parent company of Zelle), joins High Signal to discuss the evolution of high-stakes data leadership and governance. From her early work in online advertising at DoubleClick to shaping data strategy at Nike and holding Chief Data Officer roles at Bank of the West and T. Rowe Price, Cara has seen every iteration of the data leader’s role. Now, she’s navigating her 'product era'—shaping the data strategy for Early Warning's Decisions Intelligence business, where she leverages rich financial data and data science to drive fraud monitoring and modeling.

In this episode, Cara shares her pragmatic 'progress over perfection' approach to governance, why she’s abandoning monolithic platforms in favor of incremental data products, and her 80/20 rule for balancing operational rigor with innovation. We also discuss why 'loving' data isn't enough—you have to actually 'take care' of it—and why AI is finally shining a spotlight on the often-neglected fundamentals of data stewardship and conversational BI.

LINKS

Cara Dailey on LinkedIn

Why AI Adoption Fails: A Behavioral Framework for AI Implementation, A High Signal Conversation with Lis Costa (Chief of Innovation, Behavioural Insights Team)

Watch the podcast episode on YouTube

High Signal podcast

Delphina's Newsletter