Re: How to bulletproof anon FTP downloads?

From: Shao Li (chowshaoli@yahoo.com)
Date: 05/17/02


From: chowshaoli@yahoo.com (Shao Li)
Date: 17 May 2002 11:13:41 -0700

I have a situation where anonymous users are uploading files to the
ftp server. What would be the best way to ensure that these files
arrive at the ftp server intact? Is there any threat that someone
could intentionally modify the file while transit?

-- shao-li

"Alan J. Flavell" <flavell@mail.cern.ch> wrote in message news:<Pine.LNX.4.40.0205171620200.3289-100000@lxplus035.cern.ch>...
> On May 16, bill inscribed on the eternal scroll:
>
> > I need to automate the periodic downloading of certain important
> > files. Unfortunately, I have no control over the sites where these
> > files reside. I've been doing the downloads so far using anonymous
> > FTP (via a Perl script), but I have not found any way to ensure that
> > the downloaded file has not been corrupted in transit,
>
> If this is so critical, then the resource itself would need an
> engineered-in integrity check (as some archive formats have), or at
> the very least be accompanied by some kind of checksum/hash file
> throughout its distribution process (see e.g
> http://www.apache.org/dist/httpd/ ).
>
> > something that
> > happens rarely but still too frequently for my purposes. (I can check
> > for size, but all the corrupted downloads I've seen recently pass the
> > file-size test.)
>
> I can assure you, from several decades in the business, that file
> transfers don't just get randomly corrupted. There must be a cause
> somewhere. If it's more likely to get corrupted on its way to you
> than on its way from where it was generated to where it's being
> distributed or mirrored, then this should send you a message about
> something. If it was me then I'd want to work at _that_ problem, you
> know.
>
> > Ideally, I would (somehow!?) get a checksum of the remote file and
> > compare it against the downloaded file's checksum.
>
> Ideally, an authoritative checksum of the remote file would be
> provided, as I said. There are precedents (not only Apache!).
>
> If you're retrieving via some unprivileged anonymous read-only FTP,
> then the idea that you might find some way of yourself computing a
> checksum of the remote resource before it's transferred seems to be
> ill-founded.
>
> > Is there any way
> > to do this? (The only solution I can think for this is some sort of
> > MD5 signature server running on the same site as the FTP server...)
>
> If you believe that this kind of corruption is prevalent in file
> transfers (IME it's actually pretty rare, and corrupt files are
> usually a result of operational mistakes), then what's to say that the
> mirror copy that you're trying to download didn't get corrupted during
> the mirroring process? Computing an MD5 from that would be pointless.
> It needs to be computed when the file is created, and distributed
> along with it, IMHO.



Relevant Pages

  • Re: How to bulletproof anon FTP downloads?
    ... arrive at the ftp server intact? ... I've been doing the downloads so far using anonymous ... >> compare it against the downloaded file's checksum. ... > If you believe that this kind of corruption is prevalent in file ...
    (comp.security.unix)
  • Re: How to bulletproof anon FTP downloads?
    ... I've been doing the downloads so far using anonymous ... > compare it against the downloaded file's checksum. ... If you believe that this kind of corruption is prevalent in file ... Computing an MD5 from that would be pointless. ...
    (comp.security.unix)
  • Re: How to bulletproof anon FTP downloads?
    ... I've been doing the downloads so far using anonymous ... > compare it against the downloaded file's checksum. ... If you believe that this kind of corruption is prevalent in file ... Computing an MD5 from that would be pointless. ...
    (comp.security.unix)
  • Re: How to bulletproof anon FTP downloads?
    ... I've been doing the downloads so far using anonymous ... What you allege is that your FTP transfer has somehow managed to get corrupted ... _despite_ the checksum in the ethernet layer, as well as the checksum in the ... I'd advise finding _where_ the corruption is occurring and fixing it, though, ...
    (comp.security.unix)
  • Re: How to bulletproof anon FTP downloads?
    ... I've been doing the downloads so far using anonymous ... What you allege is that your FTP transfer has somehow managed to get corrupted ... _despite_ the checksum in the ethernet layer, as well as the checksum in the ... I'd advise finding _where_ the corruption is occurring and fixing it, though, ...
    (comp.security.unix)