Re: strange HTTP requests
From: Bill Carlson (wcarlson_at_vh.org)
Date: 08/26/03
- Previous message: Harlan Carvey: "Re: Outgoing connections to Port 22226 and 22227"
- In reply to: bugtraq_at_cgisecurity.net: "Re: strange HTTP requests"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]
Date: Tue, 26 Aug 2003 14:12:34 -0500 (CDT) To: incidents@securityfocus.com
On Tue, 26 Aug 2003 bugtraq@cgisecurity.net wrote:
> Yes many will, but not all. I know of people who have designed spiders for the company they work for without assigning
> a user-agent header. Most search engines on the otherhand will assign one for their spiders. I agree though it is fairly
I would also expect any spider to use robots.txt. For this traffic, no
other requests are generally seen.
Further data: The address pool of these requests is something like 6000+
IPs, scattered all over the Net. Current hit rate is hovering around 2.5
hits/second. Yes, that adds up to 7 million or so a month. My normal site
traffic is about 8-10 million a month, covering roughly 500,000 IPs.
Bill Carlson
-- Systems Administrator wcarlson@vh.org | Anything is possible, Virtual Hospital http://www.vh.org/ | given time and money. University of Iowa Hospitals and Clinics | Opinions are mine, not my employer's. | --------------------------------------------------------------------------- Attend Black Hat Briefings & Training Federal, September 29-30 (Training), October 1-2 (Briefings) in Tysons Corner, VA; the world's premier technical IT security event. Modeled after the famous Black Hat event in Las Vegas! 6 tracks, 12 training sessions, top speakers and sponsors. Symantec is the Diamond sponsor. Early-bird registration ends September 6.Visit us: www.blackhat.com ----------------------------------------------------------------------------
- Previous message: Harlan Carvey: "Re: Outgoing connections to Port 22226 and 22227"
- In reply to: bugtraq_at_cgisecurity.net: "Re: strange HTTP requests"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]
Relevant Pages
|
