When you use sftp/scp to transfer directories, you notice it ends up taking a long time. It works on files one at a time and they can often take a small amount of time to start and finish. A solution is to use tar to stream the file over ssh (so you have one continuious stream, without starting/stopping a lot).
ssh [email protected] “tar cf – big_dir/” | tar xf –
I timed transferring a 330MB directory with more than 3600 files in it.
Transferring over 10/100 Ethernet over SMBFS with ‘cp -r’
Transferring over 54mb WiFi (at 70% signal strength) using ssh/tar
Transfering over 10/100 Ethernet using ssh/tar
It took 37 less seconds over wireless (pretty impressive, especially when dealing with only 330MB). It was a full 2 MINUTES (and 8 seconds) faster over the same kind of connection.
This is useful both in a LAN enviornment and over the internet.