Re: MD5/SHA (or other hash tool) for complete directory (sub-)tree ?

From: Michael Amling (nospam_at_nospam.com)
Date: 07/04/04 

Date: Sun, 04 Jul 2004 16:27:05 GMT

Rob Warnock wrote:
> Tom St Denis <tom@securescience.net> wrote:
> +---------------
> | Rob Warnock wrote:
> | > Tom St Denis <tom@securescience.net> wrote:
> | > +---------------
> | > | md5sum `find -type f | xargs`
> | > +---------------
> | >
> | > I think you really meant:
> | > find -type f | xargs md5sum
> |
> | No, I really mean the former.
> +---------------
>
> First, if you really meant that, then as Michael Amling points out
> you have no need for "xargs" at all: "md5sum `find -type f'" is
> just as good (or bad, see below).
>
> Second, as I pointed out to Michael Amling, your version will *fail*
> (with an E2BIG error) as soon as the total number/size of the files
> exceeds the system ARG_MAX (a.k.a. NCARGS) limit, which is typically
> 65536 chars on FreeBSD (see "/usr/include/sys/syslimits.h") or 131072
> chars on Linux ("/usr/include/linux/limits.h"). At that point you
> *need* to use the form I suggested, so that no single execution of
> "md5sum" will be attempted with an argument list exceeding ARG_MAX.
> (That's why the "xargs" program even *exists*, after all!!)

   Thank you for pointing this out. You're saying that something like

     find * -type f|xargs -n 250 md5sum

will work in directories having a greater number of files than

     md5sum `find * -type f`

can handle.

--Mike Amling

Relevant Pages

  • Re: MD5/SHA (or other hash tool) for complete directory (sub-)tree ?
    ... > | Rob Warnock wrote: ... > First, if you really meant that, then as Michael Amling points out ... Personally I only use xargs when I want to make/apply patch sets for a ...
    (sci.crypt)
  • Re: Looping at toplevel?
    ... rpw3@xxxxxxxx (Rob Warnock) writes: ... saying, `don't use eval,' but they don't say why ... that talks about it online somewhere? ... thank you--and i have to play around with it for a while...) ...
    (comp.lang.lisp)