Re: robots.txt: Good, Bad, Ugly?
- From: "Barton L. Phillips" <barton@xxxxxxxxxxxx>
- Date: Sat, 23 Sep 2006 17:31:42 GMT
buck wrote:
Thank you for your reply. I think that is very interesting. It is more of a reason for me to look at implementing your "trap". I will look at my access_log files a bit more closely now and see if I see similar patterns (though it might be hard to see).
It really woudln't help because almost all of the entries in badrobots
are one shot abusers. They come to steal graphics or looking for php,
(Etc.) to abuse, get spider trapped by bad.html and are done. I don't
keep track of second attempts but as far as I can tell they never come
back.
Thanks again
--
----------------
Barton L. Phillips
Applied Technology Resources, Inc.
Tel: (818)652-9850
Web: http://www.applitec.com
.
- References:
- robots.txt: Good, Bad, Ugly?
- From: caveman
- Re: robots.txt: Good, Bad, Ugly?
- From: Dave {Reply Address In.sig}
- Re: robots.txt: Good, Bad, Ugly?
- From: Barton L. Phillips
- robots.txt: Good, Bad, Ugly?
- Prev by Date: 6 Tips for search privacy - from eff
- Next by Date: Re: robots.txt: Good, Bad, Ugly?
- Previous by thread: Re: robots.txt: Good, Bad, Ugly?
- Next by thread: Re: robots.txt: Good, Bad, Ugly?
- Index(es):
Relevant Pages
|
|
