Re: Cisco LEAP

From: johnadams (johnadams_at_apple.com)
Date: 11/03/03

  • Next message: Mike Rauch: "SQL injection question" 
    Date: Mon, 3 Nov 2003 12:56:20 -0800
To: Rob Shein <shoten@starpower.net>

    On Monday, November 3, 2003, at 11:59 AM, Rob Shein wrote:

    > It's not a question of peak performance as much as consistency. Flat
    > files
    > aren't meant to work this way; that's largely why database
    > applications work
    > the way they do in the first place. If something like paging competes
    > for
    > drive access just long enough, the whole thing can go to hell. When
    > you're
    > opening a graphic or text file completely into memory to view or edit
    > it?
    > For that, sure, a flat file is faster. But when you're streaming
    > through a
    > flat file that's dozens of gigs in size, over an extended period of
    > time
    > while running the data into a memory and processor-intensive program
    > at the
    > same time? Try it, and just see how quickly that works over the
    > length of
    > the entire file compared to a database :)

    The real issue here is the right tool for the job -- we're talking
    about a file with many passwords in it, which ostensibly would be under
    a few megabytes in size. You could mmap() the entire thing into memory
    and get consistent access without the use of a database. Memory is
    cheap these days.

    One thing that I see much of in software design is an overwhelming
    desire to put everything into a database with complete disregard for
    performance,

    I used to work at Inktomi, and we used very little in the way of
    databases to hold massive datasets (all web pages on the Internet.) We
    avoided databases for performance reasons, and saw serious gains
    because of customized code that read flat files filled with structures.

    I guess the thing to remember here is that eventually the database has
    to write your data out to disk, and when that happens, it'll be placed
    on the disk in a file, using an fwrite() and a modicum of indexes into
    the data. Even programs like mysql eventually write their data out as
    BerkeleyDB files.

    -john
    (posting far outside the scope of pen-test now)

    ---------------------------------------------------------------------------
    Network with over 10,000 of the brightest minds in information security
    at the largest, most highly-anticipated industry event of the year.
    Don't miss RSA Conference 2004! Choose from over 200 class sessions and
    see demos from more than 250 industry vendors. If your job touches
    security, you need to be here. Learn more or register at
    http://www.securityfocus.com/sponsor/RSA_pen-test_031023
    and use priority code SF4.
    ----------------------------------------------------------------------------

  • Next message: Mike Rauch: "SQL injection question"

    Relevant Pages

    • RE: Cisco LEAP
      ... he's going to want to do the kind of customizing that Inktomi did just to ... >> opening a graphic or text file completely into memory to ... >> For that, sure, a flat file is faster. ... > and get consistent access without the use of a database. ...
      (Pen-Test)
    • Re: dataset Performence Issue
      ... Microsoft that a DataSet is okay to abuse as a DataBase. ... Managed Code can never be as fast and as optimized ... very good for 90% of the situations i.e. normal memory usage, ... Merge/GetChanges - and oh lets not forget keeping your disconnected cache ...
      (microsoft.public.dotnet.framework.adonet)
    • Re: To Normalize or not ??
      ... The problem is that when you run a split database, ... save a word document (it is in memory, and thus does not get saved). ... ms-access is different then excel or word. ... database server. ...
      (microsoft.public.access.formscoding)
    • Re: TimesTen and In Memory Databases.....
      ... I have been assigned to evaluate the TimesTen in memory database ... Please note I am not a DBA but I do have quite a bit of systems ... This can hurt TimesTen performance (and Oracle ... Right - link Times Ten into your application memory, ...
      (comp.databases.oracle.server)
    • Re: Can I Trust Pointer Arithmetic In Re-Allocated Memory?
      ... and leaving the database in an undefined state. ... memory and leave data it arrived unchanged. ... have already allocated with a null pointer. ... SOME people have to clean up in that fashion. ...
      (comp.lang.c)