Re: web downoload robots

From: \ (dvader@deathstar.mil)
Date: 12/20/01


From: "\"Crash\" Dummy" <dvader@deathstar.mil>
Date: Thu, 20 Dec 2001 09:05:34 -0500


>Do anybody know how to protect web site contents against
>automat download by some robots software?

I know, but if I told you, I'd have to kill you. :-)

Seriously, I had the same problem with my web site. I don't use Linux, I am
using NT 5, but I don't think any server has an "on-off" switch to block
automated mass downloading, and pirates ignore "robots.txt" instructions. I
have to filter requests through server side scripts to block mass downloads.
I am using ASP, but PHP or Perl or any other server scripting language would
serve as well. For obvious reasons, I won't reveal exactly what my scripts
do.

--
Dave "Crash" Dummy
Certified Dilettante
Dave@gpick.com
http://lists.gpick.com