通过网络传输大量文件的方法总结

tar的方式

同设备

$ tar cf - . | (cd /dst; tar xvf -)

垮网络

$ tar czf - <files> | ssh user@host "cd /wherever && tar xvzf -"

其他工具

rsync

zsync

http://zsync.moria.org.uk/

unison

https://github.com/bcpierce00/unison

ncFTPd 非开源的

https://www.ncftp.com/ncftpd/features.html#performance

ttcp 网络反射

https://ftp.arl.mil/mike/ttcp.html

lftp

支持bittorrent, http, 等，但是没测试过 https://lftp.yar.ru/lftp-man.html

其他linux命令

1. gcp Besides of progress estimation, gcp does not copy when the file exists, skips to the next file if occurs an error, and all the fails are written to a journal file
2. pv

参考

https://www.zylk.net/en/web-2-0/blog/-/blogs/how-to-copy-files-in-linux-faster-and-safer-than-cp

https://unix.stackexchange.com/questions/10026/how-can-i-best-copy-large-numbers-of-small-files-over-scp

https://serverfault.com/questions/18125/how-to-copy-a-large-number-of-files-quickly-between-two-servers