Re: web downoload robots

From: \ (
Date: 12/20/01

From: "\"Crash\" Dummy" <>
Date: Thu, 20 Dec 2001 09:05:34 -0500

>Do anybody know how to protect web site contents against
>automat download by some robots software?

I know, but if I told you, I'd have to kill you. :-)

Seriously, I had the same problem with my web site. I don't use Linux, I am
using NT 5, but I don't think any server has an "on-off" switch to block
automated mass downloading, and pirates ignore "robots.txt" instructions. I
have to filter requests through server side scripts to block mass downloads.
I am using ASP, but PHP or Perl or any other server scripting language would
serve as well. For obvious reasons, I won't reveal exactly what my scripts

Dave "Crash" Dummy
Certified Dilettante