Archive for IWA News

A note on unsupported rules in robots.txt

Yesterday we announced that we’re open-sourcing Google’s production robots.txt parser. It was an exciting moment that paves the road for potential Search open sourcing projects in the future! Feedback is helpful, and we’re eagerly collecting questions from developers and webmasters alike. One question stood out, which we’ll address in this post:Why isn’t a code handler […]

Read More →

Posted in: IWA News

Leave a Comment (0) →

Google’s robots.txt parser is now open source

For 25 years, the Robots Exclusion Protocol (REP) was only a de-facto standard. This had frustrating implications sometimes. On one hand, for webmasters, it meant uncertainty in corner cases, like when their text editor included BOM characters in their robots.txt files. On the other hand, for crawler and tool developers, it also brought uncertainty; for […]

Read More →

Posted in: IWA News

Leave a Comment (0) →

Bye Bye Preferred Domain setting

As we progress with the migration to the new Search Console experience, we will be saying farewell to one of our settings: preferred domain. It’s common for a website to have the same content on multiple URLs. For example, it might have the same content on http://example.com/ as on https://www.example.com/index.html. To make things easier, when […]

Read More →

Posted in: IWA News

Leave a Comment (0) →

Webmaster Conference: an event made for you

Over the years we attended hundreds of conferences, we spoke to thousands of webmasters, and recorded hundreds of hours of videos to help web creators find information about how to perform better in Google Search results. Now we’d like to go further: help those who aren’t able to travel internationally and access the same information. […]

Read More →

Posted in: IWA News

Leave a Comment (0) →
Page 6 of 9 «...45678...»