Detecting and blocking bad bots
Video: Detecting and blocking bad botsIn this screencast, we are going to protect our web site against bad bots. We've seen how to do this with a plug-in in the previous screencast, but there is a better, more efficient, way to protect your site directly, using the htaccess file. Here, in our FTP/file editor, we're looking at our site's web-accessible route htaccess file. To implement this method, grab a copy of the htaccess code that's included in the exercise files of this screencast. Copy everything and then return to your file editor and paste the code beneath any existing rules.
- Next steps
Viewers: in countries Watching now:
This course explains how to secure self-hosted WordPress sites, including site configuration, code modification, and the use of free plug-ins. Beginning with the basics of site security, author Jeff Starr explains how to harden a WordPress site by configuring authentication keys, setting proper file permissions, and removing version numbers. The course shows how to implement a firewall, prevent automated spam, and control proxy access, and concludes with a series of advanced tips and site security best practices.
- Backing up and restoring your site
- Setting up strong passwords
- Choosing trusted plugins and themes
- Protecting the configuration file and the admin directory
- Securing the login page
- Fighting comment spam
- Blocking access and detecting hacks
- Finding and reporting vulnerabilities
Detecting and blocking bad bots
In this screencast, we are going to protect our web site against bad bots. We've seen how to do this with a plug-in in the previous screencast, but there is a better, more efficient, way to protect your site directly, using the htaccess file. Here, in our FTP/file editor, we're looking at our site's web-accessible route htaccess file. To implement this method, grab a copy of the htaccess code that's included in the exercise files of this screencast. Copy everything and then return to your file editor and paste the code beneath any existing rules.
This chunk of code is like a virtual control panel for blocking bad bots and user agents. First, we're blocking blank user agents. Then these lines here collectively block some of the worst known bad bots. Then this last section here is the part that actually does the blocking, based on what you have listed in these previous directives. And best of all, no upfront editing is required for this code to work.
Just save and upload the file to your server. To see it in action, let's return to the browser and visit this ridiculously handy user agent bot-simulation tool. First, let's just see it work by adding the URL of our web site and clicking the Go button. Here is our demo site that we're working with. So we copy the URL from the address bar and return to Bots versus Browsers and paste that URL here.
Then we click the Go button. As expected, our site is accessible when using the legit user agent, specified here. So now let's check that the code is working by spoofing a request from one of our blocked user agents, or blocked bots. Returning to our FTP editor. Let's grab a random user agent, skygrid, copy, and return to our Bots versus Browsers page. Paste it into the user agent field like so and then click Go with your site's URL still in the URL field. A 403: Forbidden means that the request has been blocked.
This is exactly the response we want to send to bad bots. It's is a simple response that's easy on the server. Using a plug-in would have required significantly more resources to deliver the same response, requiring WordPress, plug-in files, and the database just to block a bad bot. Using htaccess lets Apache just make the block directly at the server level, which is the optimal way of doing it. To add new bad bots to the list, return to your file editor, and we can either create a new line or just add the name to an existing line, like so.
Casing shouldn't matter, because of this directive here NoCase. So you can use any combination of upper- and lowercase letters and the result will be the same. Or we can instead just start a new line like so, sort of emulate the previous lines, and then put our new blocked bots on their own line.
And this is a good way to help keep things nice and organized, as you use this method to protect your site. That's all there is to it, so let's save and upload the file and return to our handy Bots versus Browser page to see it work. We type in the name of the user agent that we just added and click the Go button. That's it right there. Our request using this user agent has been blocked.
In this screencast, we've seen how to block bad bots and user agents from accessing our web site. Using htaccess instead of a plug-in, we're able to block bad bots directly, with greater efficiency and better site performance.
There are currently no FAQs about WordPress 3: Developing Secure Sites.