Home | Mirror | Search |
rsync is an open source utility that provides fast incremental file transfer. rsync is freely available under the GNU General Public License version 2 and is currently being maintained by Wayne Davison.
過程 73.1. rsync
安裝rsync
在AS3 第二張CD上找到rsync-2.5.6-20.i386.rpm
[root@linuxas3 root]# cd /mnt [root@linuxas3 mnt]# mount cdrom [root@linuxas3 mnt]# cd cdrom/RedHat/RPMS [root@linuxas3 RPMS]# rpm -ivh rsync-2.5.6-20.i386.rpm
配置/etc/rsyncd.conf
在rh9,as3系統上rsync安裝後,並沒有創建rsyncd.conf文檔,要自己創建rsyncd.conf文檔
[root@linuxas3 root]# vi /etc/rsyncd.conf uid=nobody gid=nobody max connections=5 use chroot=no log file=/var/log/rsyncd.log pid file=/var/run/rsyncd.pid lock file=/var/run/rsyncd.lock #auth users=root secrets file=/etc/rsyncd.passwd [postfix] path=/var/mail comment = backup mail ignore errors read only = yes list = no auth users = postfix [netkiller] path=/home/netkiller/web comment = backup 9812.net ignore errors read only = yes list = no auth users = netkiller [pgsqldb] path=/var/lib/pgsql comment = backup postgresql database ignore errors read only = yes list = no
選項說明
uid = nobody gid = nobody use chroot = no # 不使用chroot max connections = 4 # 最大連接數為4 pid file = /var/run/rsyncd.pid #進程ID檔案 lock file = /var/run/rsync.lock log file = /var/log/rsyncd.log # 日誌記錄檔案 secrets file = /etc/rsyncd.pwd # 認證檔案名,主要保存用戶密碼,權限建議設為600,所有者root [module] # 這裡是認證的模組名,在client端需要指定 path = /var/mail # 需要做鏡像的目錄 comment = backup xxxx # 註釋 ignore errors # 可以忽略一些無關的IO錯誤 read only = yes # 只讀 list = no # 不允許列檔案 auth users = postfix # 認證的用戶名,如果沒有這行,則表明是匿名 [other] path = /path/to... comment = xxxxx
密碼檔案
在server端生成一個密碼檔案/etc/rsyncd.pwd
[root@linuxas3 root]# echo postfix:xxx >>/etc/rsyncd.pwd [root@linuxas3 root]# echo netkiller:xxx >>/etc/rsyncd.pwd [root@linuxas3 root]# chmod 600 /etc/rsyncd.pwd
啟動rsync daemon
[root@linuxas3 root]# rsync --daemon
添加到啟動檔案
echo "rsync --daemon" >> /etc/rc.d/rc.local [ OK ]
cat /etc/rc.d/rc.local 確認一下
測試
[root@linux docbook]# rsync rsync://netkiller.8800.org/netkiller [root@linux tmp]# rsync rsync://netkiller@netkiller.8800.org/netkiller Password: [chen@linux temp]$ rsync -vzrtopg --progress --delete postfix@netkiller.8800.org::postfix /tmp Password:
過程 73.2. installation setp by setp
installation
$ sudo apt-get install rsync
enable
$ sudo vim /etc/default/rsync RSYNC_ENABLE=true
config /etc/rsyncd.conf
$ sudo vim /etc/rsyncd.conf uid=nobody gid=nobody max connections=5 use chroot=no pid file=/var/run/rsyncd.pid lock file=/var/run/rsyncd.lock log file=/var/log/rsyncd.log #auth users=root secrets file=/etc/rsyncd.secrets [neo] path=/home/neo/www comment = backup neo ignore errors read only = yes list = no auth users = neo [netkiller] path=/home/netkiller/public_html comment = backup netkiller ignore errors read only = yes list = no auth users = netkiller [mirror] path=/var/www/netkiller.8800.org/html/ comment = mirror netkiller.8800.org exclude = .svn ignore errors read only = yes list = yes [music] path=/var/music comment = backup music database ignore errors read only = yes list = no [pgsqldb] path=/var/lib/pgsql comment = backup postgresql database ignore errors read only = yes list = no auth users = neo,netkiller
/etc/rsyncd.secrets
$ sudo vim /etc/rsyncd.secrets neo:123456 netkiller:123456
$ sudo chmod 600 /etc/rsyncd.secrets
start
$ sudo /etc/init.d/rsync start
test
$ rsync -vzrtopg --progress --delete neo@localhost::neo /tmp/test1/ $ rsync -vzrtopg --progress --delete localhost::music /tmp/test2/
firewall
$ sudo ufw allow rsync
yum install xinetd
配置 /etc/xinetd.d/rsync
vim /etc/xinetd.d/rsync # default: off # description: The rsync server is a good addition to an ftp server, as it \ # allows crc checksumming etc. service rsync { disable = yes flags = IPv6 socket_type = stream wait = no user = root server = /usr/bin/rsync server_args = --daemon log_on_failure += USERID }
disable = yes 改為 disable = no
# vim /etc/rsyncd.conf
chkconfig xinetd on /etc/init.d/xinetd restart
# Minimal configuration file for rsync daemon # See rsync(1) and rsyncd.conf(5) man pages for help # This line is required by the /etc/init.d/rsyncd script pid file = /var/run/rsyncd.pid port = 873 address = 192.168.1.171 #uid = nobody #gid = nobody uid = root gid = root use chroot = yes read only = yes #limit access to private LANs hosts allow=192.168.1.0/255.255.255.0 10.0.1.0/255.255.255.0 hosts deny=* max connections = 5 motd file = /etc/rsyncd/rsyncd.motd #This will give you a separate log file #log file = /var/log/rsync.log #This will log every file transferred - up to 85,000+ per user, per sync #transfer logging = yes log format = %t %a %m %f %b syslog facility = local3 timeout = 300 [home] path = /home list=yes ignore errors auth users = linux secrets file = /etc/rsyncd/rsyncd.secrets comment = linuxsir home exclude = beinan/ samba/ [beinan] path = /opt list=no ignore errors comment = optdir auth users = beinan secrets file = /etc/rsyncd/rsyncd.secrets [www] path = /www/ ignore errors read only = true list = false hosts allow = 172.16.1.1 hosts deny = 0.0.0.0/32 auth users = backup secrets file = /etc/backserver.pas [web_user1] path = /home/web_user1/ ignore errors read only = true list = false hosts allow = 202.99.11.121 hosts deny = 0.0.0.0/32 uid = web_user1 gid = web_user1 auth users = backup secrets file = /etc/backserver.pas [pub] comment = Random things available for download path = /path/to/my/public/share read only = yes list = yes uid = nobody gid = nobody auth users = pub secrets file = /etc/rsyncd.secrets
$ rsync -v -u -a --delete --rsh=ssh --stats localfile username@hostname:/home/username/
for example:
I want to copy local workspace of eclipse directory to another computer.
$ rsync -v -u -a --delete --rsh=ssh --stats workspace neo@192.168.245.131:/home/neo/
rsync使用方法
rsync rsync://認證用戶@主機/模組
rsync -vzrtopg --progress --delete 認證用戶@主機::模組 /mirror目錄
transfer file from src to dest directory
neo@netkiller:/tmp$ mkdir rsync neo@netkiller:/tmp$ cd rsync/ neo@netkiller:/tmp/rsync$ ls neo@netkiller:/tmp/rsync$ mkdir src dest neo@netkiller:/tmp/rsync$ echo file1 > src/file1 neo@netkiller:/tmp/rsync$ echo file2 > src/file2 neo@netkiller:/tmp/rsync$ echo file3 > src/file3
skipping directory
neo@netkiller:/tmp/rsync$ mkdir src/dir1 neo@netkiller:/tmp/rsync$ mkdir src/dir2 neo@netkiller:/tmp/rsync$ rsync src/* dest/ skipping directory src/dir1 skipping directory src/dir2
recurse into directories
neo@netkiller:/tmp/rsync$ rsync -r src/* dest/ neo@netkiller:/tmp/rsync$ ls dest/ dir1 dir2 file1 file2 file3
backup
neo@netkiller:/tmp/rsync$ rsync -r --backup --suffix=.2008-11-21 src/* dest/ neo@netkiller:/tmp/rsync$ ls dest/ dir1 dir2 file1 file1.2008-11-21 file2 file2.2008-11-21 file3 file3.2008-11-21 neo@netkiller:/tmp/rsync$
backup-dir
neo@netkiller:/tmp/rsync$ rsync -r --backup --suffix=.2008-11-21 --backup-dir mybackup src/* dest/ neo@netkiller:/tmp/rsync$ ls dest/ dir1 dir2 file1 file1.2008-11-21 file2 file2.2008-11-21 file3 file3.2008-11-21 mybackup neo@netkiller:/tmp/rsync$ ls dest/mybackup/ file1.2008-11-21 file2.2008-11-21 file3.2008-11-21
rsync -r --backup --suffix=.2008-11-21 --backup-dir ../mybackup src/* dest/ neo@netkiller:/tmp/rsync$ ls dest mybackup src neo@netkiller:/tmp/rsync$ ls src/ dir1 dir2 file1 file2 file3
update
neo@netkiller:/tmp/rsync$ rm -rf dest/* neo@netkiller:/tmp/rsync$ rsync -r -u src/* dest/ neo@netkiller:/tmp/rsync$ echo netkiller>>src/file2 neo@netkiller:/tmp/rsync$ rsync -v -r -u src/* dest/ building file list ... done file2 sent 166 bytes received 42 bytes 416.00 bytes/sec total size is 38 speedup is 0.18
update by time and size
neo@netkiller:/tmp/rsync$ echo Hi>src/dir1/file1.1 neo@netkiller:/tmp/rsync$ rsync -v -r -u src/* dest/ building file list ... done dir1/file1.1 sent 166 bytes received 42 bytes 416.00 bytes/sec total size is 41 speedup is 0.20
--archive
rsync -a src/ dest/
--compress
rsync -a -z src/ dest/
--delete
src
svn@netkiller:~$ ls src/ dir1 dir2 file1 file2 file3
dest
neo@netkiller:~$ rsync -v -u -a --delete -e ssh svnroot@127.0.0.1:/home/svnroot/src /tmp/dest svnroot@127.0.0.1's password: receiving file list ... done created directory /tmp/dest src/ src/file1 src/file2 src/file3 src/dir1/ src/dir2/ sent 104 bytes received 309 bytes 118.00 bytes/sec total size is 0 speedup is 0.00
src
svn@netkiller:~$ rm -rf src/file2 svn@netkiller:~$ rm -rf src/dir2
dest
neo@netkiller:~$ rsync -v -u -a --delete -e ssh svnroot@127.0.0.1:/home/svnroot/src /tmp/dest svnroot@127.0.0.1's password: receiving file list ... done deleting src/dir2/ deleting src/file2 src/ sent 26 bytes received 144 bytes 68.00 bytes/sec total size is 0 speedup is 0.00
http://samba.anu.edu.au/rsync/examples.html
例 73.1. examples
用rsync刪除目標目錄
mkdir /root/blank
rsync --delete-before -a -H -v --progress --stats /root/blank/ ./cache/
例 73.2. backup to a central backup server with 7 day incremental
#!/bin/sh # This script does personal backups to a rsync backup server. You will end up # with a 7 day rotating incremental backup. The incrementals will go # into subdirectories named after the day of the week, and the current # full backup goes into a directory called "current" # tridge@linuxcare.com # directory to backup BDIR=/home/$USER # excludes file - this contains a wildcard pattern per line of files to exclude EXCLUDES=$HOME/cron/excludes # the name of the backup machine BSERVER=owl # your password on the backup server export RSYNC_PASSWORD=XXXXXX ######################################################################## BACKUPDIR=`date +%A` OPTS="--force --ignore-errors --delete-excluded --exclude-from=$EXCLUDES --delete --backup --backup-dir=/$BACKUPDIR -a" export PATH=$PATH:/bin:/usr/bin:/usr/local/bin # the following line clears the last weeks incremental directory [ -d $HOME/emptydir ] || mkdir $HOME/emptydir rsync --delete -a $HOME/emptydir/ $BSERVER::$USER/$BACKUPDIR/ rmdir $HOME/emptydir # now the actual transfer rsync $OPTS $BDIR $BSERVER::$USER/current
例 73.3. backup to a spare disk
I do local backups on several of my machines using rsync. I have an extra disk installed that can hold all the contents of the main disk. I then have a nightly cron job that backs up the main disk to the backup. This is the script I use on one of those machines. #!/bin/sh export PATH=/usr/local/bin:/usr/bin:/bin LIST="rootfs usr data data2" for d in $LIST; do mount /backup/$d rsync -ax --exclude fstab --delete /$d/ /backup/$d/ umount /backup/$d done DAY=`date "+%A"` rsync -a --delete /usr/local/apache /data2/backups/$DAY rsync -a --delete /data/solid /data2/backups/$DAY The first part does the backup on the spare disk. The second part backs up the critical parts to daily directories. I also backup the critical parts using a rsync over ssh to a remote machine.
例 73.4. mirroring vger CVS tree
The vger.rutgers.edu cvs tree is mirrored onto cvs.samba.org via anonymous rsync using the following script. #!/bin/bash cd /var/www/cvs/vger/ PATH=/usr/local/bin:/usr/freeware/bin:/usr/bin:/bin RUN=`lps x | grep rsync | grep -v grep | wc -l` if [ "$RUN" -gt 0 ]; then echo already running exit 1 fi rsync -az vger.rutgers.edu::cvs/CVSROOT/ChangeLog $HOME/ChangeLog sum1=`sum $HOME/ChangeLog` sum2=`sum /var/www/cvs/vger/CVSROOT/ChangeLog` if [ "$sum1" = "$sum2" ]; then echo nothing to do exit 0 fi rsync -az --delete --force vger.rutgers.edu::cvs/ /var/www/cvs/vger/ exit 0 Note in particular the initial rsync of the ChangeLog to determine if anything has changed. This could be omitted but it would mean that the rsyncd on vger would have to build a complete listing of the cvs area at each run. As most of the time nothing will have changed I wanted to save the time on vger by only doing a full rsync if the ChangeLog has changed. This helped quite a lot because vger is low on memory and generally quite heavily loaded, so doing a listing on such a large tree every hour would have been excessive.
例 73.5. automated backup at home
I use rsync to backup my wifes home directory across a modem link each night. The cron job looks like this #!/bin/sh cd ~susan { echo date dest=~/backup/`date +%A` mkdir $dest.new find . -xdev -type f \( -mtime 0 -or -mtime 1 \) -exec cp -aPv "{}" $dest.new \; cnt=`find $dest.new -type f | wc -l` if [ $cnt -gt 0 ]; then rm -rf $dest mv $dest.new $dest fi rm -rf $dest.new rsync -Cavze ssh . samba:backup } >> ~/backup/backup.log 2>&1 note that most of this script isn't anything to do with rsync, it just creates a daily backup of Susans work in a ~susan/backup/ directory so she can retrieve any version from the last week. The last line does the rsync of her directory across the modem link to the host samba. Note that I am using the -C option which allows me to add entries to .cvsignore for stuff that doesn't need to be backed up.
例 73.6. Fancy footwork with remote file lists
One little known feature of rsync is the fact that when run over a remote shell (such as rsh or ssh) you can give any shell command as the remote file list. The shell command is expanded by your remote shell before rsync is called. For example, see if you can work out what this does: rsync -avR remote:'`find /home -name "*.[ch]"`' /tmp/ note that that is backquotes enclosed by quotes (some browsers don't show that correctly).
#!/usr/bin/perl my $path = "/data"; #本地目錄 my $ip="172.16.xxx.xxx"; #遠程目錄 my $maxchild=5; #同時並發的個數 open FILE,"ls $path|"; while() { chomp; my $filename = $_; my $i = 1; while($i<=1){ my $un = `ps -ef |grep rsync|grep -v grep |grep avl|wc -l`; $i =$i+1; if( $un < $maxchild){ system("rsync -avl --size-only $path/$_ $ip:$path &") ; }else{ sleep 5; $i = 1; } } }