Re: challenges in capturing Gigabit ethernet



I've tried some experiments with collecting at high-speed off gigabit cards. Created a test system that generated traffic at varied rates from 10 mbs to 1 gbs. Though the cards themselves reported very little loss (Endace products) the ability to write the data to disk started failing around 250 mbs. Though I intend to continue testing in the near future I first see that I need to become a little smarter on storage and which is the quickest for write-speed.

Understand also that the tests were not done under the most scientific conditions - still repeated testing showed our breakdown was primarily the ability to write the data to disk and not the gigabit card's ability to process the data.

The next question would be, if we COULD write to disk as fast as we collected the data, at which point do other factors such as the OS, the CPU, the memory, or the aplication itself become a factor?

The testing I am doing is for a Windows-based application and, as such, I have not yet looked at the same conditions under *nix. I have some interest in that area though so may see what I can find out there as well.

I know I have seen other folks discuss these same concerns here in the past. I seem to recall someone else having found the ability to write to disk a severe limitation. Mind you that most ID/IPS system do not try to write every packet to disk - they usually only write the packet or session that that triggered an event. So, for IDS/IPS sensors any speed limitations tend to deal with how much "analysis" must be done on the traffic passing by/through the sensor. The more rules in effect - the less they are able to process. Which is a big reason why "tuning" of sensors to your network is so important. Normalizing the sensor to your particular network configuration, addressing scheme, etc. will reduce a LOT of the false positives. Disabling rules that do not apply to your network also improves the sensor's time to spend analyzing the traffic you DO determine is valid.

OK folks ... Flame on! <grin>

Hank

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------



Relevant Pages

  • I hope someone can help....CD-RW not seeing anything
    ... I recently lost all read or write ability to my CD-RW. ... Clicking on the drive gives a msg box of "please a disk in drive "E". ... was found in the drivers base and my settings for the files found bad are ... that my settings in Local Policies are not allowing many things to happen for ...
    (microsoft.public.windowsxp.general)
  • Re: Problem installing Office 2000 Professional
    ... What do you mean by collaboration ability and email/PIM? ... Disk 1 does not work on other computers. ... Is there a way to remove it from the old computer & install ... Also do you know anything about Open Office ...
    (microsoft.public.office.setup)
  • Time machine pushing up the daisies
    ... Apparently my Time machine is corrupted beyond the ability of Disk ... Utility to repair. ... DU says to just d/l any files you can and then start ...
    (comp.sys.mac.system)
  • Re: Collet fixture question
    ... then drill holes in the outer perimeter of the "threaded ... >disk". ... The purpose of this would be to have the ability to tighten the ...
    (alt.machines.cnc)
  • Re: File system performance, hardware performance, ext3, 3ware RAID1, etc.
    ... >It depends on the disk too. ... Lots of disks are far slower ... ability to reorder reads is limited in ATA to 32 tags, ... To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ ...
    (Linux-Kernel)