Re: Disabling Encryption on file transfers

From: Darren Tucker (dtucker_at_gate.dodgy.net.au)
Date: 07/24/05 

Date: 24 Jul 2005 16:56:30 GMT

On 2005-07-24, Ben Harris <bjharris@chiark.greenend.org.uk> wrote:
> In article <1122212960.838039.230040@o13g2000cwo.googlegroups.com>,
> Gushi <google@gushi.org> wrote:
>>I was transferring some (big) log files to another box on the same
>>subnet, and while the files were impressively large (5-6gigs each), I
>>found that the time it would take to transfer them was fairly slow
>>(more than 12 hours per file).
>
> Be aware that in SSH-2, about half of the CPU time used by a typical
> implementation is in the MAC function, whose purpose is to ensure that your
> files don't get modified in transit. You probably don't want to disable
> that.
>
> I'm a little surprised at your figures, incidentally. I run a machine whose
> purpose is to collect backups over the network, and it usually manages to
> copy about 30GB from its clients in two hours. It's a dual 450MHz
> UltraSPARC-II, so pretty feeble by modern standards. You aren't turning on
> compression, are you?

I second that. If you have half-decent CPUs then encryption is likely
to not be your bottleneck.

I've measured ~700 KBytes/sec scp'ing to an old SPARCstation 5 (170MHz)
which is still ~50% faster than what you're seeing.

Some suggestions:
* check the CPU usage on both hosts during transfer. If the usage isn't
high then it's probably not encryption.

* try different cipher/mac combinations (arcfour+hmac-md5-96 will probably
be the fastest of the standard ones).

* turn off compression (if you have more bandwidth than CPU) or on (if
you have more CPU than bandwidth).

* If you have a "long, fat pipe" (ie high latency, high bandwidth such as
satellite) and you're using OpenSSH then try the HPN patches below.

* try rsync -z if you're transferring files that are only partially changed.

[...]
> The SSH-2 spec allows for key re-exchanges that change the encryption type,
> so either side of the connection could renegotiate a cipher of "none" after
> authentication, as long as the other side was agreeable. I don't know of
> any implementation with this facility, mind.

The PSC OpenSSH patches implement rekey-to-none for scp:
http://www.psc.edu/networking/projects/hpn-ssh/

The vanilla OpenSSH doesn't. I'm not aware of any other implemenation that
does, either. 

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

Relevant Pages

  • Re: Why does Safari use so much processor power?
    ... The page you mention as an example is full of so much graphical crap ... Transferring, then rendering, all that "stuff" ... mostly-text web page is going to burn very little CPU. ...
    (comp.sys.mac.system)
  • Re: Fustrations with USB 2.0
    ... I suspect your speed issue may be related to one I had with transferring ... a driver problem, but when I moved to a newer computer with SATA the problem ... Serial ATA, unlike IDE, uses virtually no CPU for hard ...
    (microsoft.public.windowsxp.hardware)
  • Re: DFS Question
    ... DFS-R doesn't need much bandwidth. ... it uses a compression algorithm where only the changed bits of the files will be transferred (instead of transferring the whole file all over again). ... Microsoft MVP: Windows Server ...
    (microsoft.public.windows.server.general)
  • Re: Data Migration
    ... >bandwidth for transferring a single file, but it will put pauses ... I'm not so sure it doesn't limit bandwidth during a single file... ... the --bwlimit option to a daily 14Mbyte (single file) background transfer, ...
    (RedHat)
Quantcast