Re: Possible Error in Patch 331953?

From: Robb (robbycollins_at_hotmail.com)
Date: 05/06/03 

Date: 6 May 2003 10:56:12 -0700

Hi Jason,

We have had a very similar experience to your own. The patch was
released on 25 March. Shortly thereafter we started seeing the ASP
requests queued max out for IIS 5.0 in perfmon resulting in
"blocking". The thing that really caught my eye is that you are using
xcache, which we are also using. Although our CPU is not spiking at
100%, our sites become very unstable even when not under load. We are
using SQL Server 2000 and the db server doesn't seem to be affected at
all. Occasionally we can cause the webserver to die when we start
hitting it with a crawler such as teleport pro or win HT track. Again,
this never results in consistant errors.

Are you using RDO? We are using RDO on both our website and a desktop
app. We began seeing intermitant 4100 errors (Business object cannot
be created) in our desktop app at exactly the same time that our
website started to sporadically die.

We have contacted Microsoft and they have just supplied us with a
hot-fix which contained the folowing warning:

"WARNING: This fix is not publicly available through the Microsoft
website as it has not gone through full Microsoft regression testing.
If you would like confirmation that this fix is designed to address
your specific problem, or if you would like to confirm whether there
are any special compatibility or installation issues associated with
this fix, you are encouraged to speak to a Support Professional in
Product Support Services."

We are preparing to deploy the hotfix to our test servers to see if it
will get rid of the problem. We also intend to use IISState
(http://www.iisfaq.com/default.aspx?view=P197) to help us debug any
issues. I will let you know if we have any success. Likewise, I would
be interested to hear if you come up with something.

Regards,
R.

"Jason Martin" <jason.martin@ensonix.com> wrote in message news:<026a01c302cc$cf5f2950$a401280a@phx.gbl>...
> I've been working on a problem that was brought to my
> attention on Wednesday of last week. The original
> complaint was that our Oracle database server CPU
> Utilization went up to 100%, without high traffic on our
> website. However the real issue was that our IIS 5.0
> Services all restarted at the same time, which caused our
> caching software to start caching each server all at the
> same time.
>
> When first looking in to the problem, I was wondering
> what caused the caching software to re-cache, however it's
> supposed to do that when IIS 5.0 is restarted, and when
> reviewing the logs I found before the re-caching started,
> the services did restart. I also found out that we had
> the exact same problem one week earlier, and in the NT
> System Event logs I found the same two entries for the
> same problem. However, the issue has not ever occurred in
> the event log prior. Here is the cluster of events that
> show the IIS 5.0 Restart:
>
> Event Type: Error
> Event Source: Service Control Manager
> Event Category: None
> Event ID: 7031
> Date: 4/8/2003
> Time: 12:43:10 PM
> User: N/A
> Computer: ESXLAWEB01
> Description:
> The Site Server Authentication Service service terminated
> unexpectedly. It has done this 4 time(s). The following
> corrective action will be taken in 0 milliseconds: No
> action.
>
> Event Type: Error
> Event Source: Service Control Manager
> Event Category: None
> Event ID: 7031
> Date: 4/8/2003
> Time: 12:43:10 PM
> User: N/A
> Computer: ESXLAWEB01
> Description:
> The IIS Admin Service service terminated unexpectedly. It
> has done this 1 time(s). The following corrective action
> will be taken in 1 milliseconds: Run the configured
> recovery program.
>
> Event Type: Error
> Event Source: Service Control Manager
> Event Category: None
> Event ID: 7031
> Date: 4/8/2003
> Time: 12:43:10 PM
> User: N/A
> Computer: ESXLAWEB01
> Description:
> The Site Server LDAP Service service terminated
> unexpectedly. It has done this 4 time(s). The following
> corrective action will be taken in 0 milliseconds: No
> action.
>
> Event Type: Error
> Event Source: Service Control Manager
> Event Category: None
> Event ID: 7031
> Date: 4/8/2003
> Time: 12:43:10 PM
> User: N/A
> Computer: ESXLAWEB01
> Description:
> The Site Server LDAP Service service terminated
> unexpectedly. It has done this 4 time(s). The following
> corrective action will be taken in 0 milliseconds: No
> action.
>
> So understanding the following facts of our web
> network, which are; 12 Web Servers Running 2000 Server and
> IIS 5.0 being load balanced behind a f5 BIG/ip box, each
> server running XCache, each server running IIS5 Restart
> (configured 6 months ago and only for timed restarts), No
> Exchange or Mail Services, and 11 out of the 12 servers
> all had the exact same event log entries at the same time,
> I did the following:
>
> 1) Scanned for virus/worms (found none)
> 2) Went through the web logs looking for out of place
> requests (found none)
> 3) Looked for a change in other processes in the event log
> (found none)
> 4) Reviewed missing DoS attack Patches (found 1 web server
> missing 331953, the only server which was not effected,
> all servers which had this patch were)
>
> I believe I read a lot of Microsoft bulletins and
> newsgroups, and have not found an entry which has had the
> same symptoms, I also noted that patch 331953 was released
> 3/25/03 (and although I do not handle the updates) our
> issue first occurred on 3/28/03 and I know the patch has
> been loaded on the 11 of the 12 servers that had the issue.
>
> Has anyone had issues close to this after applying
> this patch, or is this just a coincidence?
>
> Thanks in Advanced
> Jason Martin

Relevant Pages