Search Off the Record

Episodios disponibles

5 de 56

How Googlebot Crawls the Web
In this episode of Search Off the Record, Martin and Gary from the Google Search Relations team take a deep dive into how Googlebot and web crawling work—past, present, and future. Through their humorous and thoughtful conversation, they explore how crawling evolved from the early days of the internet, when scripts could index a chunk of the web from a single homepage, to the more complex and considerate systems used today. They discuss the basics of what a crawler is, how tools like cURL or Wget relate, and how policies like robots.txt ensure crawlers play nice with web infrastructure. The conversation also covers Google's internal shift to unified infrastructure for all crawling needs, highlighting how different teams moved from separate crawlers to a shared system that enforces consistent policies. They explain why some fetches bypass robots.txt (like user-initiated actions) and the rising impact of automated traffic from new products and AI agents. With a nod to initiatives like Common Crawl, the episode ends with a look at the road ahead, acknowledging growing internet congestion but remaining optimistic about the web’s capacity to adapt. Resources: Episode transcript → https://goo.gle/sotr092-transcript Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team. #SOTRpodcast #SEO #SearchOfTheRecord Speakers: Martin Splitt, Gary Illyes Products Mentioned: Googlebotl, Gemma, Google AI
--------
32:41
Debugging the Internet: HTTP, TCP, and You
In this episode of Search Off the Record, Gary Illyes and Martin Splitt from the Google Search team dive deep into the foundations of how the web works—specifically HTTP, TCP, UDP, and newer technologies like QUIC and HTTP/3. The two reflect on how even experienced web professionals often overlook or forget the mechanics behind these core protocols, sharing insights through technical discussion, playful banter, and analogies ranging from messenger pigeons to teapots. The conversation spans key concepts like packet transmission, connection handshakes, and the importance of status codes such as 404, 204, and even 418 (“I’m a teapot”). Throughout the conversation, they connect these protocols back to real-world implications for site owners, developers, and SEOs—like why Search Console might report network errors, and how browser or server behavior is influenced by low-level transport decisions. With a mix of humor and expertise, Gary and Martin aim to demystify a crucial part of the internet's infrastructure and remind listeners of the layered complexity that makes modern web experiences possible. Resources: Episode transcript →https://goo.gle/sotr091-transcript Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team. #SOTRpodcast #SEO #Http Speakers: Lizzi Sassman, John Mueller, Martin Splitt, Gary Illyes Products Mentioned: Search Console - General
--------
33:25
Launching Search Central Live Deep Dive
In this episode of Search Off the Record, Google's Martin Splitt, Cherry, and Gary Illyes discuss the exciting expansion of Search Central Live with the introduction of Search Central Live Deep Dive. They share the history of Search Central events, the inspiration behind creating a more advanced, multi-day experience, and how community feedback helped shape the new format. Cherry explains how the Deep Dive events will offer in-depth sessions, practical workshops, and greater opportunities for networking within the Search community across the APAC region. The team also offers a behind-the-scenes look at the complexities of planning international events, from budgeting and travel logistics to ensuring high-quality attendee experiences. They highlight plans for new locations including Budapest and South Africa, and outline how interested participants can apply. If you want to hear about the future of Google's Search community events and how to get involved, this episode is a must-watch. Resources: Episode transcript → https://goo.gle/sotr090-transcript Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral
--------
35:51
How are web standards made?
Ever wondered how web standards are made? Martin and Gary from Google Search take you behind the scenes of the internet's governing bodies. From the IETF to the W3C, learn about the consensus-driven processes that shape the web. Find out why these standards are crucial for ensuring a consistent and reliable online experience. Resources: Episode transcript → https://goo.gle/sotr089-transcript Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team. #SOTRpodcast #SEO Speakers: Lizzi Sassman, John Mueller, Martin Splitt, Gary Illyes Products Mentioned: Search Console - General
--------
44:56
Wrapping up 2024
As 2024 comes to a close, we're reflecting on what SEO things happened this year. Listen along as we chat about what we learned from going to events, the importance of understanding how AI works, and whether everything will be replaced with JavaScript. Resources: Episode transcript → https://goo.gle/sotr088-transcript Encoding + Googlebot → https://goo.gle/4iZEKXO Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.
--------
37:52

Más podcasts de Tecnología

Podcasts a la moda de Tecnología

Acerca de Search Off the Record

Search Off the Record takes you behind the scenes of Google Search and its inner workings! In each episode, the folks from the Search Relations team will give you background info on the decision-making behind launches, feature prioritization in Search Console, and the projects Google Search teams are working on. They will share fun stories from the many conferences they attend as well as from their day-to-day working life at Google. They will also dive into the currently trending conversations in the SEO community at large. Have a listen!

Sitio web del podcast

Tecnología