Home | 簡體中文 | 繁體中文 | 雜文 | 打賞(Donations) | Github | OSChina 博客 | 雲社區 | 雲棲社區 | Facebook | Linkedin | 知乎專欄 | 視頻教程 | About

第 6 章 Distributed File Systems

目錄

6.1. DRBD (Distributed Replicated Block Device)
6.1.1. disk and partition
6.1.2. Installation
6.1.3. configure
6.1.4. Starting
6.1.5. Using
6.2. Network Block Device protocol
6.2.1. nbd-server - Network Block Device protocol - server
6.2.2. nbd-client - Network Block Device protocol - client
6.3. GridFS
6.3.1. nginx-gridfs
6.3.2. lighttpd-gridfs
6.4. Moose File System
6.4.1. Master server installation
6.4.2. Backup server (metalogger) installation
6.4.3. Chunk servers installation
6.4.4. Users’ computers installation
6.4.5. Testing MFS
6.5. LizardFS
6.6. Ceph
6.6.1. Installation on Ubuntu
6.6.2. Installation on CentOS
6.6.2.1. mon
6.6.2.2. mds
6.6.2.3. osd
6.6.2.4. client
6.6.2.5. RADOS Gateway
6.6.3. Block Devices
6.7. GlusterFS
6.7.1. glusterfs-server
6.7.2. glusterfs-client
6.7.3. Testing
6.7.4. RAID
6.7.4.1. Mirror
6.7.4.2. Strip
6.7.5. Filesystem Administration
6.7.6. CentOS 6.3
6.8. Lustre
6.9. MogileFS
6.10. Kosmos distributed file system (KFS)
6.11. Hadoop - HDFS
6.12. BeeGFS - The Parallel Cluster File System
6.13. Coda
6.14. OpenAFS

6.1. DRBD (Distributed Replicated Block Device)

Homepage: http://www.drbd.org/

實驗環境需要兩台電腦,如果你沒有,建議你使用VMware,並且為每一個虛擬機添加兩塊硬碟。

實驗環境

  1. master: 192.168.0.1 DRBD:/dev/sdb

  2. slave: 192.168.0.2 DRBD:/dev/sdb

6.1.1. disk and partition

Each of the following steps must be completed on both nodes

show all of disk and partition

neo@master:~$ sudo sfdisk -s
/dev/sda:   8388608
/dev/sdb:   2097152
total: 10485760 blocks
			

create a new partition on the disk /dev/sdb

$ sudo cfdisk /dev/sdb			
			

you must have extended partition

check partition

neo@master:~$ sudo fdisk -l

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000301bd

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         993     7976241   83  Linux
/dev/sda2             994        1044      409657+   5  Extended
/dev/sda5             994        1044      409626   82  Linux swap / Solaris

Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         261     2096451    5  Extended
/dev/sdb5               1         261     2096419+  83  Linux

			

format /dev/sdb1

neo@master:~$ sudo mkfs.ext3 /dev/sdb1
			

you also can using other file system

reiserfs

neo@master:~$ sudo mkfs.reiserfs /dev/sdb1		
			

I suggest you using reiserfs.

6.1.2. Installation

Each of the following steps must be completed on both nodes

search drbd8-utils package

neo@master:~$ apt-cache search drbd
drbd8-utils - RAID 1 over tcp/ip for Linux utilities
drbd0.7-module-source - RAID 1 over tcp/ip for Linux module source
drbd0.7-utils - RAID 1 over tcp/ip for Linux utilities
drbdlinks - Manages symlinks into a shared DRBD partition
			

installation

neo@master:~$ sudo apt-get install drbd8-utils			
			

to add modules from the Linux Kernel

neo@master:~$ sudo modprobe drbd
neo@master:~$ lsmod |grep drbd
drbd                  213000  0
cn                      9632  1 drbd
			
			

6.1.3. configure

Each of the following steps must be completed on both nodes

backup configure file

			
neo@master:~$ sudo cp /etc/drbd.conf /etc/drbd.conf.old
			
			

edit /etc/drbd.conf

global { 
  usage-count yes; 
}
common {
  protocol C;
}
resource r0 {
  on master {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   192.168.0.1:7789;
    meta-disk internal;
  }
  on slave {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   10.1.1.32:7789;
    meta-disk internal;
  }
}
			

6.1.4. Starting

Each of the following steps must be completed on both nodes.

neo@master:~$ sudo drbdadm create-md r0	
neo@master:~$ sudo drbdadm attach r0
neo@master:~$ sudo drbdadm connect r0
neo@master:~$ sudo drbdadm -- --overwrite-data-of-peer primary r0

neo@slave:~$ sudo drbdadm create-md r0		
neo@slave:~$ sudo drbdadm attach r0
neo@slave:~$ sudo drbdadm connect r0

			

master

			
neo@master:~$ sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

slave

			
neo@slave:~# sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

status

neo@master:~$ cat /proc/drbd
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43
 0: cs:StandAlone st:Primary/Unknown ds:UpToDate/DUnknown   r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
        resync: used:0/31 hits:0 misses:0 starving:0 dirty:0 changed:0
        act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0
 1: cs:Connected st:Secondary/Secondary ds:Diskless/Inconsistent C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
			
			

6.1.5. Using

master

neo@master:~$ sudo drbdadm primary all		
neo@master:~$ sudo mkfs.reiserfs /dev/drbd0
neo@master:~$ sudo mkdir /mnt/drbd0
neo@master:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@master:~$ sudo touch /mnt/drbd0/helloworld.tmp
neo@master:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             7.6G  1.3G  6.0G  18% /
varrun                125M  216K  125M   1% /var/run
varlock               125M  8.0K  125M   1% /var/lock
udev                  125M   60K  125M   1% /dev
devshm                125M     0  125M   0% /dev/shm
/dev/drbd0            2.0G   33M  2.0G   2% /mnt/drbd0
neo@master:~$ sudo dd if=/dev/zero of=/mnt/drbd0/tempfile1.tmp bs=104857600 count=1
1+0 records in
1+0 records out
104857600 bytes (105 MB) copied, 0.564911 s, 186 MB/s
neo@master:~$ sudo umount /mnt/drbd0/
neo@master:~$ sudo drbdadm secondary all
			

slave

neo@slave:~$ sudo drbdadm primary all			
neo@slave:~$ sudo mkdir /mnt/drbd0
neo@slave:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@slave:~$ ls /mnt/drbd0/
helloworld.tmp  tempfile1.tmp