Search Console URL Inspect does not want your junk

Search Console’s URL Inspection Tool Does Not Want Your Junk

This is niche enough that I’m almost embarrassed to post it, but by adding this to the industry knowledge I may save someone else 15 minutes of frustration.

If you perform a fresh URL Inspection in Google Search Console for a live asset that isn’t a page, Google Search Console will claim it’s blocked in robots.txt.

If you want to see how an image is being interpreted right now, you can’t. Search Console will say it’s blocked in robots.txt and refuse to fetch it.

Try it yourself.

Request an image in URL inspect:

Search Console's URL Inspection Tool Does Not Want Your Junk 1

Now request the live URL:

Search Console's URL Inspection Tool Does Not Want Your Junk 2

Now get told it’s blocked in robots.txt:

Search Console's URL Inspection Tool Does Not Want Your Junk 3

Use the link to test if it’s blocked in robots.txt to confirm it isn’t blocked in robots.txt:

Search Console's URL Inspection Tool Does Not Want Your Junk 4

Search Console's URL Inspection Tool Does Not Want Your Junk 5

Now waste far too long on trying to solve this.

Check your logs to confirm it’s being crawled (it is). Try and solve the cause and discover something else far more useful in the process.

Repeat on multiple websites to confirm it’s just how Search Console Works for most things that aren’t pages or redirects.

  • If it’s a redirect, Google will assess the status and filetype of the destination.
  • If it’s a 404 or another error, Google will abandon the page.
  • There are some other spooky things here but I’m saving them for ohgmcon.

If you changed a supposedly “blocked in robots.txt” URL to a redirect, it becomes crawlable suggesting that the URL isn’t really blocked, and this message is a shorthand for a hardcoded “Oliver, please stop clogging Google Search Console with this shit.”:

Search Console's URL Inspection Tool Does Not Want Your Junk 6

And this makes sense – this tool is about the indexing of documents, not stuff.

But sometimes this stuff makes it into Search Console Coverage reports.

Why Would I Care?

Previously you could submit whatever you wanted for indexing through Search Console’s Fetch & Render

Search Console's URL Inspection Tool Does Not Want Your Junk 7

Search Console's URL Inspection Tool Does Not Want Your Junk 8
Not that I’m sure this does anything, and I can’t recall ever having done it.

You might miss this functionality, even if it was placebo.

Hopefully the error messaging here is updated soon to reflect reality.

2 thoughts on “Search Console’s URL Inspection Tool Does Not Want Your Junk”

  1. I ran into this on an image a few days ago, was running around trying to diagnose a robots.txt issue that was obviously not there!

  2. You just saved me 15 or more on …car insurance… I mean time. Thank you SEO thought leader.

Leave a Reply

Your email address will not be published. Required fields are marked *