Join Brad Batesole for an in-depth discussion in this video What is Screaming Frog?, part of Learning SEO Tools.
- [Instructor] The process of Google, and other search engines evaluating your site is done through what's known as the crawl. A bot will arrive on your site, crawl your links, and hopefully index your content. The truth is though, we don't have a lot of insight into what's going on when Google and others crawl your site. Fortunately, there's a tool called Screaming Frog SEO Spider that will run on your PC, Mac, or Linux machine and crawl your site just as a search engine would.
The program will fetch all your on-site elements and then organizes the information into easy-to-understand tabs and reports. This allows you to identify common SEO issues, evaluate site changes, and perform a fairly comprehensive audit without the need to manually review every page on your site. Screaming Frog will gather everything from your URL, page titles, meta descriptions, headings, images, and so on.
It'll also check your redirects, and help you build site maps and investigate problems with your robots.txt file. This is a fantastic tool, and it's used by professional SEOs all over. The ability to automate a crawl and quickly identify problems is invaluable. The program will crawl 500 pages for free. The professional edition includes an unlimited URL crawl, and unlocks advanced features. Now it will set you back 99 pounds for a one-year license, which at the time of this recording, was about $157 US.
A good SEO is going to charge that for about an hour of their time, so it's a valuable investment if you ask me. Now I've already gone ahead and downloaded the application, but you can do that by visiting screamingfrog.co.uk. From here, you'll choose SEO Spider in the upper right-hand corner, and you'll find the download links along the right-hand side of the screen. Simply pick the one that matches your platform. I'm going to go ahead and open up the tool and give you a quick tour of the interface.
Now, I've already gone ahead and ran a crawl so that you can see how it looks once you start using it. We're going to look at configuring and running a crawl however in the next movie. Here on the right-hand side, you'll see an overview of the crawl, and this will update in realtime as your crawler is working. This will match the filters along the top of the screen. You can see those here labeled Internal, External, Protocol, Page Titles, Meta Keywords, and so on, and so forth. You can also see a little graph in the bottom right-hand corner, and if you select into that, it'll drive you to those various elements.
Here in the upper right-hand corner, you can switch between the Overview, the Site Structure, and the Response Time views. You can also Tab into any of these statistics. For example, if I want to see just my heading one tags, I'll select H1, and now I can use the table view to navigate the pages that have my heading one tags, as well as reviewing those tags, their length, and any additional heading one tags that I'm using on the page. Now, good SEO will stick to just one heading one tag, so if you're seeing multiple heading one tags, you know that you've got an issue.
You can also tap into any URL and you'll see data appear at the bottom of the screen. This allows you to review details about that particular URL, any Inlinks, Outlinks, images associated with that URL, and you can even get a view of the search engine result page, letting you know what this might look like in the Google search results. You'll also notice here in the upper left-hand corner, you have some filters, and these will be specific to the page that you're currently viewing. A tremendous amount of information can be gleaned through Screaming Frog.
I recommend downloading and setting it up on your computer, as we'll go through configuring and running our crawl together in the next movie.
- Setting SEO goals
- Configuring Google Search Console
- Evaluating the Google index
- Using PageSpeed Insights
- Interpreting results and leveraging reports
- Using the Screaming Frog SEO Spider
- Generating a site map
- Getting started with Keyword Planner
- Multiplying keyword lists
- Structuring data
- Using SEMrush
- Conducting keyword research