I've taken steps to make retarded search engines, from valid sources or spamming sources, work for me. If you take the time to log and examine your 404 errors, you can do this too.
I use WordPress as my blogging platform. There are two WordPress plugins I use to take advantage of 404 errors produced by retarded search engines. If you use a different platform, there may be plugins for it as well. If not, you can do it with your server log files and an ".htaccess" file, it's just harder.
Examining the 404 errors
The WordPress plugin I use to log 404 errors is called "Redirection". Some people have problems with it, but I'm not one of them. The plugin logs 404 errors, successful redirections and allows me to add redirections without having to mess with my htaccess file all the time.
The redirections can be literal URLs or regular expressions. Using regular expressions is the most powerful part of redirections and they work in htaccess files as well as this plugin. They're entered a little differently with the plugin and that's just because the plugin has to be able to read them before executing them.
While examining 404 errors, I saw my real URLs with things like "/Untwisted Vortex/", "/untwisted-vortex/", "/untwisted/" and other people's domains tacked onto the ends of them. I don't know what retarded search engine does that, but it doesn't matter. I redirected the invalid URLs to the valid ones using a regular expression redirection.
I also saw invalid URLs pointing to the translated versions that were created by the Wordpress Global Translator Plugin, a plugin I removed more than 6 months ago (see "My Global Translator Plugin Is Out Of Here!"). I was able to successfully redirect the invalid URLs to the valid URLs using another regular expression redirection.
Generating a 404 error of my own on purpose
What about the real nonexistent pages the retarded search engines are generating errors for? One of them has been trying to index "http://www.untwistedvortex.com/google-search/" for months. Guess what? It's just a template file!
A couple of days ago, I pulled up the Redirection plugin and entered this:
Source: http://www.untwistedvortex.com/google-search/
Target: http://www.untwistedvortex.com/toyota-altis/
Yes, it generates a brand new 404 error and redirects to my 404 error page, but it's not just any error page. It's an error page enhanced by the AskApache Google 404 WordPress plugin. The article, "SEO Traffic Boost Confirmed from 404 Plugin", is absolutely correct. I checked last night and I ranked number 7 on Google for that term. The redirection has happened over 400 times!
Some of my regular expression redirections
Here are just 3 of over a hundred, and they all work.
Source: http://www.untwistedvortex.com/(.*)/Untwisted%20Vortex/
Target: http://www.untwistedvortex.com/$1/Source: http://www.untwistedvortex.com/de/(.*)/
Target: http://www.untwistedvortex.com/$1/Source: http://www.untwistedvortex.com/(.*)/digg.com
Target: http://www.untwistedvortex.com/$1/
My recommendation
As you can see, I'm a strong proponent of redirecting invalid URLs. They can't hurt your search engine rankings; they can only boost your rankings or at least put them where they belong.
You should take the time to examine your own website's 404 errors. Not only can it help your site get indexed properly, but you can find real errors in the process. Why wait for retarded search engines to index your site with a bunch of invalid URLs? Nip it in the bud and make retarded search engines work for you.
