Google To Crawl Pages For Mobile Usability

By Tinuiti Team

Google sent an email to Google Advertisers today, advising them of a mobile crawl update. Below is the initial email, and here are the main elements highlighted for Advertisers:

Google To Flag Mobile Unfriendly URLs

Google is crawling advertiser pages with new mobile bots to ensuring that the quality is just as good for mobile as it is for desktop. If Google crawl’s your page and it’s not optimized for mobile, Google is going to flag it.

Here is the email body [bolding added for emphasis]:


Thank you for your participation in Google Shopping. In today’s multi-device world, consumers use a variety of screens in the course of their online Shopping experience. Beginning on April 28, we plan to start crawling product landing pages submitted to Google Shopping for mobile devices in order to verify mobile accessibility and ensure a positive user experience across devices.

We are reaching out today to make you aware of this change. In a recent review of your Merchant Center account(s) (xxxxxxx), we’ve determined that some of your items might be impacted by this change.


In addition to crawling for desktop browsers, we will start crawling your product landing pages (as specified by the ‘link’ attribute in the feed) with the following mobile user-agent:

Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X)
AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e
Safari/8536.25 (compatible; Googlebot/2.1; +

If we detect crawling errors while fetching either the mobile or desktop landing page of an item, we will disapprove the item for both mobile and desktop devices until we are able to access the landing page successfully.


We have detected that some of your items had errors when we fetched them with a mobile user-agent. The most common errors are:

Page not found (404): The landing page was not found on the server. Please make sure to specify a proper URL of the landing page with the ‘link’ attribute.

Could not connect / HTTP 5xx response: When we tried to access the landing page the server was not available (e.g. because of maintenance work). Please make sure that your website is always available.

Hostname not resolvable: We were not able to resolve the hostname. Please make sure that your DNS configuration is setup properly.

You can find the most recent examples in the Data Quality tab of Merchant Center. You can also test your current configuration with the Fetch as Google tool described at Please make sure to select Smartphone as the type of fetch.

If you have fixed these issues and updated your items via a new feed upload or the Content API, the errors you see in the Data Quality tab should disappear within a couple of days. This time allows us to verify that we can access the landing pages that you submitted. Once this verification is complete, the errors will disappear from the Data Quality tab and your items will start serving on Google Shopping again. You can also speed up the process by increasing Google’s crawl rate (

Additionally, the information below contains helpful resources and best practices for your mobile site:


The Google Shopping Team

© 2014 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043″

Avoid Mobile Crawl Penalties

Google’s crawl update only flag’s mobile use issues so your pages doesn’t have to be optimized (or even look good) on mobile-they just need to load in some fashion.

You can view URL errors and warnings for mobile in the Google Merchant login:

Google mobile crawl errors


Here is a look at what those notifications look like:

Google ULR load for moblile

Google notes common URL issues include:


Check out this resource for additional Google data feed issues.


You Might Be Interested In

*By submitting your Email Address, you are agreeing to all conditions of our Privacy Policy.