Join Brad Batesole for an in-depth discussion in this video Evaluate the Google index, part of Learning SEO Tools.
- [Instructor] Moving along to the reports, let's take a minute to look at the Google index section. In this section, you have access to index status, blocked resources, and remove URLs. We're going to start here on the index status page. And this section provides valuable data on how many pages Google has indexed over the past year. Now, keep in mind, the number in your index is typically lower than all of the pages on your site. This can be due to a couple of reasons. First, you might have content blocked in your robots.txt file Or you might have content that is only available to users after they log in.
If you're using both secure and non-secure versions of your site, the http and https we talked about earlier than you'll only see the index for the domain that you're currently viewing. Which you're looking for is an up and to the right line suggesting that Google is continuing to gather more and more content. Now if you don't publish any new content regularly, you may see a flat line which is completely fine. If you notice a downward trend, or a complete drop to the bottom you know that you have a problem and it's time to investigate the issue further.
You can also use the Advance tab to take a closer look at what's going on by bringing in an additional view that shows you what content might be blocked by your robots file or if you select the check box here at the top of the screen. You can also see any content that you chose to be removed. Here we can see some issues where apparent with content blocked by the robots. But this drops off and the issue is resolved. Here we can also see a note line. And if I select the note Google's going to indicate that there was an issue.
So use this section to see if Google is warning you about any blocked resources. These will typically be blocked because of your robots.txt file disallowing crawling these resources. But be the judge here. Sometimes Google will indicate an error that you disagree with. It's okay. Having an error here doesn't mean that you're site is being penalized. It just means there's a potential chance it could be. Now some of the resources you find are going to be from a third party platform. Say you're using a tracking software for your analytics, or a tracking software for your ads, some of those resources maybe blocked by that company themselves.
But let's say you tell Google to stop crawling but the content isn't leaving the index. You can ask Google to update or remove the page using a removal request. Now, this is only going to temporarily remove the URL from the search result. It won't be instant and it's not permanent. To completely resolve the issue, you'll have to remove the content or the pages yourself and then make sure that Google doesn't come back and recrawl or re-index that content within say, about 90 days. Now, when you do delete a page or remove the content, make sure that the server returns a 404 error.
A hard 404 tells Google the page is done and it's not coming back. You can also use the no index meta tag to suggest to Google not to index content even if you want it displayed to users. You'll select the temporarily hide button and enter in the URL on your site that you'd like to hide and Google will give you information on the status of that request. Use the Remove URLs option if you need to purge content from the cache or remove links that are broken in Google search. You may also have published an article, removed it, and Google just hasn't updated its index yet and so you can come in here and get it pulled potentially faster.
- Setting SEO goals
- Configuring Google Search Console
- Evaluating the Google index
- Using PageSpeed Insights
- Interpreting results and leveraging reports
- Using the Screaming Frog SEO Spider
- Generating a site map
- Getting started with Keyword Planner
- Multiplying keyword lists
- Structuring data
- Using SEMrush
- Conducting keyword research