RE: [fw-wiz] tests about latency
From: Paul Robertson (proberts_at_patriot.net)
To: Neale Banks <firstname.lastname@example.org> Date: Fri, 12 Sep 2003 07:12:05 -0400 (EDT)
On Fri, 12 Sep 2003, Neale Banks wrote:
> Ob FW: Whilst obviously anything that's not simply routed (e.g. proxied
> protocols) would be a completely different kettle of fish, to what extent
> could one then reasonably generalise the results obtained from ping tests
> (i.e. ICMP packets) to other protocols?
Not well at all, ICMP handling is generally a special case for most
stacks/devices. Also, some devices prioritize different protocols, and
ICMP datagrams tend to be self-contained, not requiring things like
sequence number checking.
Finally, the size of the ICMP datagram can make a huge difference even on
the same platform, as can the type- how the stack is written may affect
which ICMP type codes get handled more quickly, for instance.
I wouldn't base any conclusions of overall performance, let alone
per-protocol performance on just an ICMP test.
Performance testing is difficult to get right, and the numbers change for
most devices with minor changes to the packets you're generating. Sizes,
fragments, windows for TCP, and the like all make different devices do
different things, _especially_ if you're trying to make a security
decision based upon the packets. For instance, how many out of sequence
packets will a device buffer before making the other end retransmit
packets? Are those buffers packet-size specific? If we fill up a
different sized buffer, will it affect overall performance for the other
buffers, and how?
The best you can hope to do is get a representative sample of traffic out
of wherever you want to put the device, then recreate a similar mix and
test with each piece. Everything else is a guess, and probably a poorly
educated one unless you completely understand the characteristics of the
hardware, stack and testing going on.
Paul D. Robertson "My statements in this message are personal opinions
email@example.com which may have no basis whatsoever in fact."
firstname.lastname@example.org Director of Risk Assessment TruSecure Corporation
firewall-wizards mailing list