parsing very large tcpdump files

From: Tom (digitalevidence_at_excite.com)
Date: 11/19/04

  • Next message: Eric McCarty: "RE: need your help about IPS and IDS,thanks" 
    To: focus-ids@securityfocus.com
Date: Thu, 18 Nov 2004 18:29:32 -0500 (EST)

    moderator: sorry if this is vague. My requirements are not fixed yet and will probably change from case to case, therefore I am just looking for generic info now.

    I was wondering if anyone on this list can recommend some tools (Opensource or commercial) to automate the parsing of very large (many GB) tcpdump files. I am trying to put together a generic toolset but in general some things I'd like to do are:

    1. Filter out traffic to/from a specific IP address or range
    2. Reconstruct all reconstructable sessions in an easy to parse way: emails, web sites visited (and content uploaded/downloaded), voip, anything else imaginable.
    3. Be able to search all of this data for keywords.

    This may seem like a tall order. I know of a few tools to do individual tasks on a small scale, such as mailsnarf, vomit, ethereal, etc. but it's not practical to use ethereal to parse these by hand. I've tried chaosreader.pl but it bogs down on files as small as 200 MB.

    I'd appreciate any input.

    Thanks.

    _______________________________________________
    Join Excite! - http://www.excite.com
    The most personalized portal on the Web!

    --------------------------------------------------------------------------
    Test Your IDS

    Is your IDS deployed correctly?
    Find out quickly and easily by testing it with real-world attacks from
    CORE IMPACT.
    Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
    to learn more.
    --------------------------------------------------------------------------

  • Next message: Eric McCarty: "RE: need your help about IPS and IDS,thanks"