Bots hitting my web server?

From: Adam Bultman (adamb@glaven.org)
Date: 08/28/02


Date: Wed, 28 Aug 2002 16:54:05 -0400 (EDT)
From: Adam Bultman <adamb@glaven.org>
To: <incidents@securityfocus.com>

Question, fellas.

I know of two boxes that had apache running on them. Apache 1.3.9, if I'm
not mistaken, with mod_proxy enabled. As a result, they were exploited
and used by someone/something to fetch pages from remote servers. In many
cases, ads (like service.bfast.com, etc) but in most cases, porn. Of
course porn. Anyway, the night I actually found out, whomever was using
the servers to fetch every single webcam image from spotlife, and using
all of my 1 MBit connection to the internet from those two servers. In my
own defense, I did not set these servers up.

Anyway, I obviously closed the hole on those servers ASAP, and was clearly
logging all data (ip, referrer, etc), expecting the users, as they noticed
things were broken, to stop using them. If I closed down the apache
server, the requests would halt rather quickly.. But once apache was
restarted, they would come back in a torrent of requests. Well, since all
they get is a 2k error page, it didn't consume my bandwidth.

Well, a few months has gone by, and there are still requests. From all
over. Canada, the US, other countries, all over. I've called ISPs. I've
written abuse lines. I've done everything I can think of to track down who
is causing this, and excatly how. Clearly, I've decided (maybe, an hour
after I noticed the traffic) that it was robots doing the hitting, but my
question is, what kind of site/robot whatever uses a proxy to just sit and
hit webcam pages? Porn pages? Ads? The clients (of which were are
literally thousands) keep coming back, but getting denied. Is this from
zombies out on the net? Companies who are referring to my servers for
their ads? What's going on? I no longer mind as much since they don't
hog my bandwidth, but I still don't know why I get so many hits, and why
to this day, if I tail my access logs, they still scroll past quite
rapidly.

Any help would be nice. Thanks.

Adam

-- 
Adam Bultman

---------------------------------------------------------------------------- This list is provided by the SecurityFocus ARIS analyzer service. For more information on this free incident handling, management and tracking system please see: http://aris.securityfocus.com



Relevant Pages

  • Re: Apache and Tux running together
    ... my job we've got a web based product provided by Apache running PHP ... This web application is hosted by multiple servers ... and MySQL totalling 15 Megs of ram), ...
    (comp.lang.php)
  • FreeBSD 6 Jails - REJ apache processes? [was: Apache 2 in 6.0 jails: Connection refused: connect
    ... Sorry to insist, really, but this bug is really annoying: today, two more apache servers have frozen while being scanner by a crawler: ... I did the same for sockstat and netstat -a, each time before and after the apache restart: ... Connection refused: connect to listener on 0.0.0.0:80 [Sat Jul ...
    (freebsd-questions)
  • Re: Warning: robots.txt unreliable in Apache servers
    ... > Subject: Warning: robots.txt unreliable in Apache servers ... server has to do is deliver the robots.txt file to the bot. ... I use Mozilla as my Web browser because I want a browser that ...
    (comp.infosystems.www.authoring.html)
  • Re: bill gates claim about security vulnerabilities per LOC in Unix versus Windows
    ... > of how their systems work. ... the most common systems in use, but didn't the latest NetCraft survey ... we can agree that the absolute populations of ISS and Apache servers are ...
    (SecProg)
  • Re: apache httpd performance
    ... Both servers share same dns server, so I do not think that the DNS can be issue. ... We'd be better off ignoring the Linux results entirely and simply try instead to optimize your FreeBSD performance. ... My big surprise was that CPU: AMD Athlon64 Processor 3200+ can't handle 100 concurrent requests normally e.g. the load on the servers goes very high, and the server does not respond very well during the test. ...
    (freebsd-performance)