Agentic Conversations (formally mlops.community)

What happens when AI agents start relying on the same messy docs your team already struggles to search?

Demetrios hops into a Waymo in San Francisco with Dave Nunez, founder and CEO of Falconer and co-author of Documentation for Developers. They talk about why developers and agents need different things from docs, why agents sometimes skip up-to-date instructions, and how a stale internal document can contribute to a production incident.

Dave shares how Stripe cut roughly 500,000 internal docs down to 5,000 worth maintaining. The conversation also covers how to capture the reasoning behind engineering decisions, when a diagram beats a wall of text, and who owns the fallout when an agent acts on bad information.

Timestamps:
[00:00] Hopping into a Waymo in San Francisco
[00:24] Meet Dave, founder of Falconer
[01:15] Why internal docs matter for AI agents
[03:24] What humans and agents each need from docs
[05:46] Testing how well agents use developer tools
[07:01] Why agents skip the latest docs
[09:42] Decision records and the reasoning behind the code
[14:04] Cleaning up stale docs and broken search
[15:29] How Stripe went from 500,000 docs to 5,000
[17:02] When a diagram beats a 10-page doc
[18:53] How conflicting docs can lead to an incident
[20:42] Who owns reliability when an agent makes a mistake?
[24:01] Treating documentation as part of the product
[26:38] Giving readers an escape hatch
[27:27] The cost of AI-generated fluff
[29:36] Wrap-up and post-show chat

Sam Herring’s first real test of Hermes Agent was a shopping tool that matched clothes to photos of ancient pottery. A few months later, the same agent was checking government auction sites every day until it found him a retired three-wheel parking ticket car, which he now drives around San Francisco. In this episode, recorded at AGNTCon + MCPCon North America in San Jose, Sam, who works on the RL post-training team at Nous Research, goes from those everyday wins to the question everyone in AI is asking: can agents improve themselves?

He explains how Hermes grew from an internal productivity tool into one of the most talked-about open-source agents, and why letting it write its own skills was the moment it clicked for users. He also describes an early version that could design its own RL environment, train on Thinking Machines’ Tinker, and hot-swap the new weights into a model running locally. Sam then breaks down Trace to Environment, an idea he credits to Xiaomi’s MiMo-V2.6-Pro work, and the loop Nous is building around it: find the exact point where an agent fails, cluster those failures by type, synthesize environments that recreate them, and train on them again and again.

The second half moves from training to inference. We get into whether reasoning models overthink, why leaning on pretraining knowledge can be a model’s Achilles heel, and why choosing a reasoning effort level may be a temporary quirk. Sam also makes his case for activation steering as an underexplored way to get more out of small models, explains where steering meets interpretability, and describes what true continual learning could look like.

Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmSam Herring: https://www.linkedin.com/in/sam-herring-67225819a

Timestamps:[0:00] Cold open[1:21] Meet Sam from Nous Research[2:39] How Hermes Agent started[3:38] From pottery to a parking ticket car[7:40] Benchmarking life admin tasks[8:58] Agents that write their own skills[10:10] RL at Nous a year ago[12:24] Hot swapping self-trained weights[15:00] Xiaomi and Trace to Environment[17:22] Turning agent failures into training[20:27] Where the RSI conversation is heading[21:34] Token efficiency and overthinking[26:30] Steering models at inference time[32:35] Toward true continual learning[34:54] Wrap up

What does AI really cost once thousands of people inside a company start using it? Ioannis Zempekakis, Director of AI at Prosus, has been running agents in production since 2024. We put that question to him on a tandem bike ride from a favorite Amsterdam cafe to Vondelpark.

Ioannis explains the decision to stop relying purely on cloud APIs, and why that approach breaks down at the scale of a large company. He covers why agent harnesses and sandboxes are as greedy as software gets, and why agents scheduled to run every five minutes repeat a lesson the data world already learned about freshness. He also explains how the habit of throwing away a codebase and prompting again quietly adds up. His bigger point is that even if model progress froze tomorrow, there would still be about two years of value left to squeeze out. The cheapest wins come from controlling context and running a basic model router, while ten new features a month keep sprouting like Hydra heads.

From there we get into when fine-tuning small models pays off for repetitive work like data analytics, and the eval suites that come as a byproduct of building in-house. Ioannis explains how owning the stack let Prosus grow a GPT-3 Slack bot into a full agent harness. He also explains why committing to a single vendor or open source project is a bigger bet than it looks, and how to balance top-down priorities with bottom-up building across the company. We close on hiring. Ioannis looks for system design over syntax, a nose for the over-engineered code AI models love to produce, and enough agent intuition to know when to build something yourself and when to let the agent decide.

Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm
Ioannis Zempekakis: https://www.linkedin.com/in/zempekakis

Timestamps:
[0:00] AI total cost of ownership on a bike
[0:48] The million dollar decision
[4:26] When everyone can build with AI
[6:51] Which use cases are worth the tokens
[10:46] Benchmarking every model on every task
[12:18] Total cost of ownership and AI ROI
[15:12] Context control and model routing
[16:39] Training and fine-tuning your own models
[19:41] Is the juice worth the squeeze
[25:26] Optionality and vendor lock-in
[27:30] From Slack bot to agent harness
[30:12] Top-down vs bottom-up AI adoption
[32:19] What AI engineer even means
[35:13] What knowing AI really means
[37:47] The hardest part of hiring

What does it take to build AI agents that deliver useful, reliable work?

Stanford lecturer and Monaco Head of AI Mihail Eric joins Demetrios for a conversation inside a Waymo about how AI is changing software development - and what it takes to put agents into production.

Mihail explains why he rebuilt most of his Modern Software Developer course, how he teaches students to avoid shipping AI slop, and why research, planning, and verification still matter as coding agents become more capable.

They discuss Claude Code, agent loops, context engineering, memory and compaction, and how to design MCP tools around what an agent needs to accomplish. Then they go inside Monaco’s approach to sales agents, from turning meetings and emails into useful insights to building a custom agent harness.

Along the way: lessons from Boris and Lee Robinson’s guest lectures, why company traces are becoming valuable assets, and Monaco’s unusual approach to brand building.

What’s the biggest bottleneck in your agent workflows: context, tools, or verification? Let us know in the comments.

Timestamps:
[00:00] Meet Mihail Eric
[00:16] Teaching AI Coding at Stanford
[02:42] Avoiding AI Slop
[04:48] Research, Plan, Build, Test
[06:19] Verifying Agent Work
[09:16] Lessons from Claude Code’s Creator
[12:40] Proactive AI Assistants
[16:13] Agent Memory & Context
[17:41] Rethinking Git for Agents
[19:32] AI Startup Pitches
[22:51] Why Traces Matter
[24:09] Better MCP Tools
[26:01] Monaco’s Sales Agents
[29:40] Turning Conversations into Insights
[33:33] AI Meets Human Expertise
[35:03] Poker & Brand Building
[36:46] Inside Monaco’s Agent Stack
[38:44] Designing Tools Around Intent
[40:12] Building an AI Advantage
[42:53] Wrapping Up

Two people, one tandem bike, and a lot of near misses along Amsterdam’s canals. Dexter Horthy, CEO and co-founder of HumanLayer and the person who coined “context engineering,” climbs on the back to talk about a problem every AI-heavy engineering team eventually hits: code that runs today that nobody understands three months later.

The ride starts with an old joke from the Claude 3.7 days: ship the slop, it’ll be GPT 7’s problem. That joke is aging badly now that GPT 6 is out and still can’t design clean abstractions. From there it becomes a tour of how agents are actually used right now. Computer use checks in for flights and books government appointments. Two hours of raw footage became a rough but promising product demo through Opus 4.8 and the Palmier MCP while Dex was out for a run. And the best MCP tools turn out to be the ones that mirror what Claude Code and Codex were trained to call.

Off the bike, over pour-over coffee, the questions get bigger. Are browser agents a bandaid until everything goes headless? What happens to ad-funded websites and upsells when an agent does the shopping? Why does chat still feel like such a low-bandwidth way to work with models? And why is every principal engineer quietly building their own software factory, then getting buried in the sync, streaming and approval plumbing that HumanLayer wants to turn into building blocks, the way GitHub Actions did for CI?

Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm
Dexter Horthy: https://www.linkedin.com/in/dexterihorthy

Timestamps
[00:00] Intro
[01:38] The GPT 7 will fix it joke
[03:03] GPT 6 still writes spaghetti
[06:21] If it runs, does it matter
[07:28] Computer use that fills out your forms
[11:57] Editing a demo with Palmier MCP
[13:16] Why most MCP servers are bad
[14:32] Mirror the tools models were trained on
[16:57] Hill climbing a demo while out running
[23:49] Are browser agents a band-aid
[32:23] Why chat needs richer interfaces
[40:30] Every engineer is building a software factory
[45:13] The GitHub Actions moment for agents