Re: Restrict by UserAgent

Yeah, I was just thinking about the post I made in early morning and
realized that URLScan isn't exactly a valid solution even if it is

Stitching together the two code samples I have will be sufficient, and it is
specifically tuned to just this situation. There won't be any gotchas or
performance issues because it is doing exactly what you need and nothing

In other words, your performance concerns about filtering all requests for
UserAgent is odd because... in order to restrict by UserAgent, don't you
have to filter all requests for it *anyway*, so even if there is a perf
concern doing this, don't you have to accept it if you want to filter?

Performance concerns for the Apache solution is real because in that case,
it is a general-purpose module being reconfigured for this task, which
carries along with it a real performance caveat due to how it functions.
However, I would not automatically assume that the same caveat affects IIS
because it doesn't, especially with a specialized module and centralized

This posting is provided "AS IS" with no warranties, and confers no rights.

"Ed" <Ed@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
> Hi again David,
> Well, after going through the available documentation on URLScan 2.5, it
> doesn't look like it'll serve this purpose at all. My interpretation of
> the
> documentation for the [DenyHeaders] settings is that it takes in a header
> NAME and not a header VALUE nor a header NAME/VALUE entry.
> In other words, it'll accept a setting like: "User-Agent:"
> but not these:
> "User-Agent: Java/1.5.0_02" ; name/value
> "Java/1.5.0_02" ; value only
> Again this is only based on my understanding/perception of the
> [DenyHeaders]
> section of URLScan ini - couldn't really find any related
> documentation/samples....
> Addtionally, the "User-Agent:" setting really seems like a disastrous
> setting in my book since it translates to "deny all requests that contain
> a
> User-Agent header" - equates to probably all known browsers. If this
> interpretation is accurate, it's somewhat counter intuitive to IIS -
> what's
> IIS for otherwise*?
> *I do remember having to install IIS on a SQL server just to create a
> certificate and have the option to secure SQL traffic (SSL) - may not have
> been the only way to do this, but after wasting a few hours trying to get
> this to work without using the IIS gui for this purpose, well, it breaks
> down
> to "just do it"...
> -----------
> Cheers,
> Ed
> "David Wang [Msft]" wrote:
>> Sure, you can do this by either:
>> 1. Use an existing module to do this
>> 2. Write some custom module to do this
>> FYI: Apache cannot do this without using a custom module which happens to
>> be
>> widely distributed with it. Similarly, IIS can also cannot do this
>> without
>> using a custom module, but no one really provides a freely distributed
>> one.
>> Some that may work are from:
>> Personally, I would use URLScan since it is an existing, supported, and
>> available module. It is really not different than how you do it on
>> Apache.
>> With Apache, you are simply configuring some pre-bundled module -- so how
>> is
>> that different than configuring a self-selected URLScan module on IIS,
>> other
>> than you having the freedom of choosing the URLScan module over any
>> other?
>> For those interested in custom code, you should be able to stitch
>> together
>> code between these two blog entries to do it.
>> - Pick out requests based on a header (user-agent:)
>> - Deny requests based on a header (referer:)
>> --
>> //David
>> IIS
>> This posting is provided "AS IS" with no warranties, and confers no
>> rights.
>> //
>> "Ed" <Ed@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
>> news:FF9735F2-E9D8-415B-A177-BF1637F9E6FA@xxxxxxxxxxxxxxxx
>> > What was a "trivial occurrence" is now a cause for concern. We've been
>> > seeing
>> > increased activity which seem to be from Java based
>> > crawlers/spiders/scrapers.
>> >
>> > User agent is of the form: Java/[various versions]
>> >
>> > Is there a way to restrict/deny requests by a specific useragent on IIS
>> > 6
>> > /W2K3 **without URLScan**?
>> >
>> > I've found references for doing so on other platfroms, particularly
>> > Apache,
>> > but so far none on IIS. Additionally, it seems that Java
>> > useragents/bots
>> > are
>> > enough of a concern that even Google has recognized such and does
>> > exactly
>> > what we're attempting to do (denies requests).
>> >
>> > Thanks to anyone who can provide guidance on this. I hope that a
>> > solution
>> > is
>> > available and can be shared with all IIS admins.
>> >
>> > -----------
>> > Cheers,
>> > Ed