Does Google respect the URL parameters tool?

Does Google respect the URL parameters tool?

E-commerce sites should not make assumptions about Google’s crawling parameter – check the log files to confirm activity.

Any e-commerce site is probably familiar with the “URL Parameters Tool.” This is a feature in Google Search Console that SEOs have long used to help control the crawl of their websites. In this tool, you inform Google of what your different URL parameters do and how Google should crawl them (“Let Googlebot Decide”, “No URLs” etc). Google has provided extensive documentation on the different settings that can be configured and how the crawl commands interact with each other. 

However, recently Google has moved this tool to the ambiguous “Legacy tool and reports” section. Ever since that time, I’ve wondered what that meant for the tool. Is this just a way of categorizing an older feature? Does Google plan on sunsetting it eventually? Does Google even still use the commands here? 

Something else I’ve found interesting is that when reviewing client log files, we’ve encountered examples where Google didn’t appear to be abiding by the rules set in the URL parameter tool. 

To find out more, I decided to perform a test. I took one of our test sites and found URL parameters that Google was crawling. Using Google’s Index Coverage report, I was able to confirm Googlebot was crawling the following parameters:

?cat
?utm_source
?utm_medium
?utm_campaign
?ref

On June 26, I went ahead and added these URLs to Google’s URL parameters report. I instructed Googlebot specifically to crawl “No URLs.” 

I then waited and monitored Google’s crawl of the site. After collecting a couple of week’s worth of data, we can see that Google was still crawling these URL parameters. The primary parameter we were able to find activity on was “?cat” URLs:

Zooming out a bit further, you can see that these are verified Googlebot events that occurred on June 27 or later, after the crawl settings had been configured: 

We were also able to confirm crawl activity of both “?cat” and “?utm” URLs using Google’s URL Inspection Tool. Notice how the URLs had “Last crawls” after the new rules went into place. 

What does this mean for SEOs? 

While we’re not seeing overwhelming crawl activity, it is an indicator that Google might not always respect the rules in the URL Parameters tool. Keep in mind that this is a smaller site (around 600 pages) so the scale in which these URL parameters will be crawled is much lower than a large eCommerce site.

Of course, this isn’t to say that Google is always ignoring the URL parameters report. However, in this particular instance, we can see that it might be the case. If you’re an e-commerce site, I would recommend not making assumptions about how Google’s crawling your parameters and check the log files to confirm crawl activity. Overall, if you’re looking to limit the crawl of a particular parameter, I’d rely on the robots.txt first and foremost.



Best SEO Company in Sri Lanka

ABOUT THE AUTHOR

Chris Long

Chris Long is a Technical SEO at Go Fish Digital who works with unique problems and advanced search situations. He helps clients improve organic traffic through a deep understanding of Google’s algorithm and Web technology.

Read More
user July 20, 2020 0 Comments

Hello world!

Welcome to WordPress. This is your first post. Edit or delete it, then start writing!

Read More
user June 19, 2020 1 Comment

66 Sources of Content Inspiration

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments

The New AdWords Experience

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments

Is Bing Really Rendering AngularJs?

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments

There are no secrets to success. It is the result of preparation, hard work, and learning from failure.

- Paul Tournier

Content Ideation: Tools & Techniques

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments

Is Bing Really Rendering NodeJs?

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments

The Simple iframe Hack

How To Fetch & Render (Almost) Any Site

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments

Good Music for Good SEO

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments

Super Quick Content Gap Analysis

The time has finally come. You’ve poured your blood, sweat, and tears into your most recent content piece, and it’s ready to be packaged up and sent to the client to be pushed live. After a few final checks and only…

Read More
user November 16, 2018 2 Comments
Close Bitnami banner
Bitnami