Podcasts · Search Off the Record
Crawling smarter, not harder
· 40 min · S1 E79
Episode Crawling smarter, not harder
In this episode of SOTR, John Mueller, Lizzi Sassman, and Gary Illyes talk about misconceptions around crawl frequency and site quality, what's challenging about crawling the web nowadays, and how search engines could crawl more efficiently.
Resources: Episode transcript → https://goo.gle/sotr079-transcript Gary's post on LinkedIn → https://goo.gle/3YAT55q Crawling episode with Dave Smart → https://goo.gle/3WShUsf If-Modified-Since → https://goo.gle/3ywXvja About the IETF → https://goo.gle/3SGVVlo Robots Exclusion Protocol → https://goo.gle/4dgmBSg Proposal for new kind of chunked transfer → https://goo.gle/3AgMF1c
Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral
Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.
#SOTRpodcast
Links in the notes 7
Transcript
Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them. Explicit shows are for adult profiles only, and children’s profiles hear only children’s shows. More about this list.