The Pragmatic Engineer

• Measuring the impact of AI on software engineering – with Laura Tacho

  Supported by Our Partners•⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more.• Graphite — The AI developer productivity platform.—There’s no shortage of bold claims about AI and developer productivity, but how do you separate signal from noise?In this episode of The Pragmatic Engineer, I’m joined by Laura Tacho, CTO at DX, to cut through the hype and share how well (or not) AI tools are actually working inside engineering orgs. Laura shares insights from DX’s research across 180+ companies, including surprising findings about where developers save the most time, why devs don’t use AI at all, and what kinds of rollouts lead to meaningful impact.We also discuss: • The problem with oversimplified AI headlines and how to think more critically about them• An overview of the DX AI Measurement framework• Learnings from Booking.com’s AI tool rollout• Common reasons developers aren’t using AI tools• Why using AI tools sometimes decreases developer satisfaction• Surprising results from DX’s 180+ company study• How AI-generated documentation differs from human-written docs• Why measuring developer experience before rolling out AI is essential• Why Laura thinks roadmaps are on their way out• And much more!—Timestamps(00:00) Intro(01:23) Laura’s take on AI overhyped headlines (10:46) Common questions Laura gets about AI implementation (11:49) How to measure AI’s impact (15:12) Why acceptance rate and lines of code are not sufficient measures of productivity(18:03) The Booking.com case study(20:37) Why some employees are not using AI (24:20) What developers are actually saving time on (29:14) What happens with the time savings(31:10) The surprising results from the DORA report on AI in engineering (33:44) A hypothesis around AI and flow state and the importance of talking to developers(35:59) What’s working in AI architecture (42:22) Learnings from WorkHuman’s adoption of Copilot (47:00) Consumption-based pricing, and the difficulty of allocating resources to AI (52:01) What DX Core 4 measures (55:32) The best outcomes of implementing AI (58:56) Why highly regulated industries are having the best results with AI rollout(1:00:30) Indeed’s structured AI rollout (1:04:22) Why migrations might be a good use case for AI (and a tip for doing it!) (1:07:30) Advice for engineering leads looking to get better at AI tooling and implementation (1:08:49) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:• AI Engineering in the real world• Measuring software engineering productivity• The AI Engineering stack• A new way to measure developer productivity – from the creators of DORA and SPACE—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:11:30

  --------

  1:11:30
• Amazon, Google and Vibe Coding with Steve Yegge

  Supported by Our Partners•⁠ WorkOS — The modern identity platform for B2B SaaS.•⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more.•⁠ Sonar — Code quality and code security for ALL code.—Steve Yegge⁠ is known for his writing and “rants”, including the famous “Google Platforms Rant” and the evergreen “Get that job at Google” post. He spent 7 years at Amazon and 13 at Google, as well as some time at Grab before briefly retiring from tech. Now out of retirement, he’s building AI developer tools at Sourcegraph—drawn back by the excitement of working with LLMs. He’s currently writing the book Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond.In this episode of The Pragmatic Engineer, I sat down with Steve in Seattle to talk about why Google consistently failed at building platforms, why AI coding feels easy but is hard to master, and why a new role, the AI Fixer, is emerging. We also dig into why he’s so energized by today’s AI tools, and how they’re changing the way software gets built.We also discuss: • The “interview anti-loop” at Google and the problems with interviews• An inside look at how Amazon operated in the early days before microservices  • What Steve liked about working at Grab• Reflecting on the Google platforms rant and why Steve thinks Google is still terrible at building platforms• Why Steve came out of retirement• The emerging role of the “AI Fixer” in engineering teams• How AI-assisted coding is deceptively simple, but extremely difficult to steer• Steve’s advice for using AI coding tools and overcoming common challenges• Predictions about the future of developer productivity• A case for AI creating a real meritocracy • And much more!—Timestamps(00:00) Intro(04:55) An explanation of the interview anti-loop at Google and the shortcomings of interviews(07:44) Work trials and why entry-level jobs aren’t posted for big tech companies(09:50) An overview of the difficult process of landing a job as a software engineer(15:48) Steve’s thoughts on Grab and why he loved it(20:22) Insights from the Google platforms rant that was picked up by TechCrunch(27:44) The impact of the Google platforms rant(29:40) What Steve discovered about print ads not working for Google (31:48) What went wrong with Google+ and Wave(35:04) How Amazon has changed and what Google is doing wrong(42:50) Why Steve came out of retirement (45:16) Insights from “the death of the junior developer” and the impact of AI(53:20) The new role Steve predicts will emerge (54:52) Changing business cycles(56:08) Steve’s new book about vibe coding and Gergely’s experience (59:24) Reasons people struggle with AI tools(1:02:36) What will developer productivity look like in the future(1:05:10) The cost of using coding agents (1:07:08) Steve’s advice for vibe coding(1:09:42) How Steve used AI tools to work on his game Wyvern (1:15:00) Why Steve thinks there will actually be more jobs for developers (1:18:29) A comparison between game engines and AI tools(1:21:13) Why you need to learn AI now(1:30:08) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:•⁠ The full circle of developer productivity with Steve Yegge•⁠ Inside Amazon’s engineering culture•⁠ Vibe coding as a software engineer•⁠ AI engineering in the real world•⁠ The AI Engineering stack•⁠ Inside Sourcegraph’s engineering culture—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:33:41

  --------

  1:33:41
• What is a Principal Engineer at Amazon? With Steve Huynh

  Supported by Our Partners•⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more.• Graphite — The AI developer productivity platform. • Augment Code — AI coding assistant that pro engineering teams love.—Steve Huynh spent 17 years at Amazon, including four as a Principal Engineer. In this episode of The Pragmatic Engineer, I join Steve in his studio for a deep dive into what the Principal role actually involves, why the path from Senior to Principal is so tough, and how even strong engineers can get stuck. Not because they’re unqualified, but because the bar is exceptionally high.We discuss what’s expected at the Principal level, the kind of work that matters most, and the trade-offs that come with the title. Steve also shares how Amazon’s internal policies shaped his trajectory, and what made the Principal Engineer community one of the most rewarding parts of his time at the company.We also go into: • Why being promoted from Senior to Principal is one of the hardest jumps in tech• How Amazon’s freedom of movement policy helped Steve work across multiple teams, from Kindle to Prime Video• The scale of Amazon: handling 10k–100k+ requests per second and what that means for engineering• Why latency became a company-wide obsession—and the research that tied it directly to revenue• Why companies should start with a monolith, and what led Amazon to adopt microservices• What makes the Principal Engineering community so special • Amazon’s culture of learning from its mistakes, including COEs (correction of errors) • The pros and cons of the Principal Engineer role• What Steve loves about the leadership principles at Amazon• Amazon’s intense writing culture and 6-pager format • Why Amazon patents software and what that process looks like• And much more!—Timestamps(00:00) Intro(01:11) What Steve worked on at Amazon, including Kindle, Prime Video, and payments(04:38) How Steve was able to work on so many teams at Amazon (09:12) An overview of the scale of Amazon and the dependency chain(16:40) Amazon’s focus on latency and the tradeoffs they make to keep latency low at scale(26:00) Why companies should start with a monolith (26:44) The structure of engineering at Amazon and why Amazon’s Principal is so hard to reach(30:44) The Principal Engineering community at Amazon(36:06) The learning benefits of working for a tech giant (38:44) Five challenges of being a Principal Engineer at Amazon(49:50) The types of managing work you have to do as a Principal Engineer (51:47) The pros and cons of the Principal Engineer role (54:59) What Steve loves about Amazon’s leadership principles(59:15) Amazon’s intense focus on writing (1:01:11) Patents at Amazon (1:07:58) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:•⁠ Inside Amazon’s engineering culture—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:13:17

  --------

  1:13:17
• How AI is changing software engineering at Shopify with Farhan Thawar

  Supported by Our Partners•⁠ WorkOS — The modern identity platform for B2B SaaS.•⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more.• Sonar —  Code quality and code security for ALL code. —What happens when a company goes all in on AI?At Shopify, engineers are expected to utilize AI tools, and they’ve been doing so for longer than most. Thanks to early access to models from GitHub Copilot, OpenAI, and Anthropic, the company has had a head start in figuring out what works.In this live episode from LDX3 in London, I spoke with Farhan Thawar, VP of Engineering, about how Shopify is building with AI across the entire stack. We cover the company’s internal LLM proxy, its policy of unlimited token usage, and how interns help push the boundaries of what’s possible.In this episode, we cover:• How Shopify works closely with AI labs• The story behind Shopify’s recent Code Red• How non-engineering teams are using Cursor for vibecoding• Tobi Lütke’s viral memo and Shopify’s expectations around AI• A look inside Shopify’s LLM proxy—used for privacy, token tracking, and more• Why Shopify places no limit on AI token spending • Why AI-first isn’t about reducing headcount—and why Shopify is hiring 1,000 interns• How Shopify’s engineering department operates and what’s changed since adopting AI tooling• Farhan’s advice for integrating AI into your workflow• And much more!—Timestamps(00:00) Intro(02:07) Shopify’s philosophy: “hire smart people and pair with them on problems”(06:22) How Shopify works with top AI labs (08:50) The recent Code Red at Shopify(10:47) How Shopify became early users of GitHub Copilot and their pivot to trying multiple tools(12:49) The surprising ways non-engineering teams at Shopify are using Cursor(14:53) Why you have to understand code to submit a PR at Shopify(16:42) AI tools' impact on SaaS (19:50) Tobi Lütke’s AI memo(21:46) Shopify’s LLM proxy and how they protect their privacy(23:00) How Shopify utilizes MCPs(26:59) Why AI tools aren’t the place to pinch pennies(30:02) Farhan’s projects and favorite AI tools(32:50) Why AI-first isn’t about freezing headcount and the value of hiring interns(36:20) How Shopify’s engineering department operates, including internal tools(40:31) Why Shopify added coding interviews for director-level and above hires(43:40) What has changed since Spotify added AI tooling (44:40) Farhan’s advice for implementing AI tools—The Pragmatic Engineer deepdives relevant for this episode:• How Shopify built its Live Globe for Black Friday• Inside Shopify's leveling split• Real-world engineering challenges: building Cursor• How Anthropic built Artifacts—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  47:03

  --------

  47:03
• The present, past and future of GitHub

  Supported by Our Partners•⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more.• Graphite — The AI developer productivity platform. • Augment Code — AI coding assistant that pro engineering teams love—GitHub recently turned 17 years old—but how did it start, how has it evolved, and what does the future look like as AI reshapes developer workflows?In this episode of The Pragmatic Engineer, I’m joined by Thomas Dohmke, CEO of GitHub. Thomas has been a GitHub user for 16 years and an employee for 7. We talk about GitHub’s early architecture, its remote-first operating model, and how the company is navigating AI—from Copilot to agents. We also discuss why GitHub hires junior engineers, how the company handled product-market fit early on, and why being a beloved tool can make shipping harder at times.Other topics we discuss include:• How GitHub’s architecture evolved beyond its original Rails monolith• How GitHub runs as a remote-first company—and why they rarely use email • GitHub’s rigorous approach to security• Why GitHub hires junior engineers• GitHub’s acquisition by Microsoft• The launch of Copilot and how it’s reshaping software development• Why GitHub sees AI agents as tools, not a replacement for engineers• And much more!—Timestamps(00:00) Intro(02:25) GitHub’s modern tech stack(08:11) From cloud-first to hybrid: How GitHub handles infrastructure(13:08) How GitHub’s remote-first culture shapes its operations(18:00) Former and current internal tools including Haystack(21:12) GitHub’s approach to security (24:30) The current size of GitHub, including security and engineering teams(25:03) GitHub’s intern program, and why they are hiring junior engineers(28:27) Why AI isn’t a replacement for junior engineers (34:40) A mini-history of GitHub (39:10) Why GitHub hit product market fit so quickly (43:44) The invention of pull requests(44:50) How GitHub enables offline work(46:21) How monetization has changed at GitHub since the acquisition (48:00) 2014 desktop application releases (52:10) The Microsoft acquisition (1:01:57) Behind the scenes of GitHub’s quiet period (1:06:42) The release of Copilot and its impact(1:14:14) Why GitHub decided to open-source Copilot extensions(1:20:01) AI agents and the myth of disappearing engineering jobs(1:26:36) Closing—The Pragmatic Engineer deepdives relevant for this episode:• AI Engineering in the real world• The AI Engineering stack•  How Linux is built with Greg Kroah-Hartman•  Stacked Diffs (and why you should know about them)•  50 Years of Microsoft and developer tools—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:27:14

  --------

  1:27:14

Más podcasts de Noticias

Podcasts a la moda de Noticias

Acerca de The Pragmatic Engineer