Any marketer focused on SEO will tell you that it’s sometimes frustrating to wait for Googlebot to recrawl an updated page. The reason is simple. Until Googlebot recrawls the page, the old content will show up in the search results. For example, imagine someone added content that shouldn’t be on a page, and that new content was already indexed by Google. Since it’s an important page, you don’t want to take the entire page down. In a situation like this, you would typically update the page, resubmit your xml sitemap, and hope Googlebot stops by soon. As you can guess, that doesn’t make anyone involved with the update very happy.
For some companies, Googlebot is visiting their website multiple times per day. But for others, it could take weeks to get recrawled. So, how can you make sure that a recently updated page gets into Google’s index as quickly as possible? Well, Google has you covered. There’s a tool called Fetch as Googlebot that can be accessed within Google Webmaster Tools that you can use for this purpose. Let’s explore Fetch as Googlebot in greater detail below.
Fetch as Googlebot and Submit to Index
If you aren’t using Google Webmaster Tools, you should be. It’s an incredible resource offered by Google that enables webmasters to receive data directly from Google about their verified websites. Google Webmaster Tools also includes a number of valuable tools for diagnosing website issues. One of the tools is called Fetch as Googlebot. The primary purpose of Fetch as Googlebot is to submit a URL, and then view how the page looks to Googlebot (Google’s web crawler). This can help you diagnose issues with the URL at hand. For example, is Googlebot not seeing the right content, is the wrong header response code being thrown, etc.
But, that’s not the only use for Fetch as Googlebot. In 2011, Google added functionality for submitting that URL to its index, right from the tool itself. You can submit up to 500 URL’s per week via Fetch as Googlebot, which should be sufficient for most websites. This can be a great solution for times where you updated a webpage and want that page refreshed in Google’s index as quickly as possible. In addition, Google provides an option for submitting the URL and linked pages to the index. This enables you to have the page at hand submitted to the index, but also other pages that are linked to from that URL. You can do this up to 10 times per month, so make sure you need it if you use it!
Let’s go through the process of using Fetch as Googlebot to submit a recently updated page to Google’s index. I’ll walk you step by step through the process below.
How to Use Fetch as Googlebot to Submit a Recently Updated Page to Google’s Index
1. Access Google Webmaster Tools and Find “Fetch as Googlebot”
You need a verified website in Google Webmaster Tools in order to use Fetch for Googlebot. Sign into Google Webmaster Tools, select the website you want to work on, expand the left side navigation link for “Diagnostics”. Then click the link for “Fetch as Googlebot”.
2. Enter the URL to Fetch
You will see a text field that begins with your domain name. This is where you want to add the URL of the page you want submitted to Google’s index. Enter the URL and leave the default option for Googlebot type as “Web”, which will use Google’s standard web crawler (versus one of its mobile crawlers). Then click “Fetch”.
3. Submit to Index
Once you click Fetch, Google will fetch the page and provide the results below. At this point, you can view the status of the fetch and click through that status to learn more. But, you’ll notice another option next to the status field that says, “Submit to Index”. Clicking that link brings up a dialog box asking if you want just the URL submitted or the URL and Linked Pages. Select the option you want and then click Submit.
A Successful Fetch:
The Submit to Index Dialog Box:
4. Submit to Index Documented with Date and Time:
Once you click “Submit”, Google will present a message that your URL has been submitted to the index, along with the date and time it was submitted.
That’s it! You just successfully added an updated page to Google’s index.
Note, this doesn’t mean the page will automatically be updated in the index. It can take a little time for this to happen, but I’ve seen this happen pretty quickly (sometimes in just a few hours). The update might not happen as quickly for every website, but again, it should be quicker than waiting for Googlebot to recrawl your site.
Expedite Updates Using Fetch as Googlebot
Let’s face it, nobody like waiting. And that’s especially the case when you have updated content that you want indexed by Google! If you have a page that’s been recently updated, then I recommend using Fetch as Googlebot to make sure the page gets updated as quickly as possible. It’s easy to use, fast, and can also be used to submit all linked URL’s from the page at hand. Go ahead, try it out today.
GG
