A Smarter Way To Block Referer Spam

Referer Spam is the bane of anyone whose site has become popular, and been noticed by spam robots.

“When you click a hyperlink on one web site, your browser passes to the next site the address of the page where you clicked the link. This is logged by the server hosting the next web site.

The referer information can be faked very easily. Some unscrupulous web site owners will arrange to have several computers access a particular web site with a referer that lists their own web site address. There are a number of ways to accomplish this […] but the result is that the web server logs of the targeted site will contain hundreds or possibly thousands of entries with the fake referer information. This is known as “referer spamming” – SpywareInfo

A lot of people attempt to block referer spam by using mod_rewrite rules in their .htaccess file to stop specific cases like :

  • A particular REFERER eg:
    RewriteEngine On
    RewriteCond %{HTTP_REFERER} ^ [NC]
    RewriteRule .* – [F,L]
  • or a specific HOST eg:
    RewriteEngine On
    RewriteCond %{HTTP_HOST} arcor-ip.net$ [NC]
    RewriteRule .* – [F,L]

This is fine, but means that each time a new REFERER or HOST decides to spam your site, you have to manually add a new rule to your .htaccess file, which is not at all practical.

A better way is to try and see patterns in the information left behind by referer spammers, and create a rule that matches and blocks visitors to your site who fit that pattern.

An example is the recent pattern of the referer spam domains that have been randomly visiting this website with URL’s like:

  • http://www.texan-holdem-win.info/
  • http://www.herbal-phentermine-appetite-suppressant-pharmacy-2003.info/
  • http://www.phentermine-shipped-cod-top-deals.info/
  • http://www.loose-slot-machines-gambling-4u.info/

The pattern is easy to spot and repeated:

  1. http://www.
  2. 1+ character a-z
  3. – character
  4. 1+ character a-z
  5. – character
  6. 1+ character a-z
  7. – character
  8. etc …

Whats good about this rule is that very few (if any) websites contain multiple dashes in their domain name.

My decision was to make the .htaccess rule match a domain which contains at least 2 dashes because I examined my logs and there are legitamate sites which contain 1 dash in their domain, but none that had 2 or more dashes.

So the rule to block all visitors to your site whose REFERER contains a domain with 2+ dashes in its name is:

See comments for a better rule than the one below

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^(http://www.)[a-z]+-[a-z]+- [NC]
RewriteRule .* – [F,L]

Further Reading

3 Replies to “A Smarter Way To Block Referer Spam”

  1. I have been experimenting further and found an even better rule than the one in the article:

    RewriteEngine On
    RewriteCond %{HTTP_REFERER} ^(http://www.)[a-z]+-[a-z]+- [NC]
    RewriteRule ^(.*) http://%{REMOTE_ADDR}/ [R=301,L]

    The main improvements is that if a computer attempts to referer spam your website, the attempt gets redirected back to itself 🙂

  2. Pingback: shirtrat(dot)net

Comments are closed.