Home | 簡體中文 | 繁體中文 | 雜文 | 知乎專欄 | Github | OSChina 博客 | 雲社區 | 雲棲社區 | Facebook | Linkedin | 視頻教程 | 打賞(Donations) | About
知乎專欄多維度架構 微信號 netkiller-ebook | QQ群:128659835 請註明“讀者”

第 72 章 Distributed File Systems

目錄

72.1. DRBD (Distributed Replicated Block Device)
72.1.1. disk and partition
72.1.2. Installation
72.1.3. configure
72.1.4. Starting
72.1.5. Using
72.2. Network Block Device protocol
72.2.1. nbd-server - Network Block Device protocol - server
72.2.2. nbd-client - Network Block Device protocol - client
72.3. GridFS
72.3.1. nginx-gridfs
72.3.2. lighttpd-gridfs
72.4. Moose File System
72.4.1. Master server installation
72.4.2. Backup server (metalogger) installation
72.4.3. Chunk servers installation
72.4.4. Users’ computers installation
72.4.5. Testing MFS
72.5. LizardFS
72.6. Ceph
72.6.1. Installation on Ubuntu
72.6.2. Installation on CentOS
72.6.2.1. mon
72.6.2.2. mds
72.6.2.3. osd
72.6.2.4. client
72.6.2.5. RADOS Gateway
72.6.3. Block Devices
72.7. GlusterFS
72.7.1. glusterfs-server
72.7.2. glusterfs-client
72.7.3. Testing
72.7.4. RAID
72.7.4.1. Mirror
72.7.4.2. Strip
72.7.5. Filesystem Administration
72.7.6. CentOS 6.3
72.8. Lustre
72.9. MogileFS
72.10. Kosmos distributed file system (KFS)
72.11. Hadoop - HDFS
72.12. BeeGFS - The Parallel Cluster File System
72.13. Coda
72.14. OpenAFS

72.1. DRBD (Distributed Replicated Block Device)

Homepage: http://www.drbd.org/

實驗環境需要兩台電腦,如果你沒有,建議你使用VMware,並且為每一個虛擬機添加兩塊硬碟。

實驗環境

  1. master: 192.168.0.1 DRBD:/dev/sdb

  2. slave: 192.168.0.2 DRBD:/dev/sdb

72.1.1. disk and partition

Each of the following steps must be completed on both nodes

show all of disk and partition

neo@master:~$ sudo sfdisk -s
/dev/sda:   8388608
/dev/sdb:   2097152
total: 10485760 blocks
			

create a new partition on the disk /dev/sdb

$ sudo cfdisk /dev/sdb			
			

you must have extended partition

check partition

neo@master:~$ sudo fdisk -l

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000301bd

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         993     7976241   83  Linux
/dev/sda2             994        1044      409657+   5  Extended
/dev/sda5             994        1044      409626   82  Linux swap / Solaris

Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         261     2096451    5  Extended
/dev/sdb5               1         261     2096419+  83  Linux

			

format /dev/sdb1

neo@master:~$ sudo mkfs.ext3 /dev/sdb1
			

you also can using other file system

reiserfs

neo@master:~$ sudo mkfs.reiserfs /dev/sdb1		
			

I suggest you using reiserfs.

72.1.2. Installation

Each of the following steps must be completed on both nodes

search drbd8-utils package

neo@master:~$ apt-cache search drbd
drbd8-utils - RAID 1 over tcp/ip for Linux utilities
drbd0.7-module-source - RAID 1 over tcp/ip for Linux module source
drbd0.7-utils - RAID 1 over tcp/ip for Linux utilities
drbdlinks - Manages symlinks into a shared DRBD partition
			

installation

neo@master:~$ sudo apt-get install drbd8-utils			
			

to add modules from the Linux Kernel

neo@master:~$ sudo modprobe drbd
neo@master:~$ lsmod |grep drbd
drbd                  213000  0
cn                      9632  1 drbd
			
			

72.1.3. configure

Each of the following steps must be completed on both nodes

backup configure file

			
neo@master:~$ sudo cp /etc/drbd.conf /etc/drbd.conf.old
			
			

edit /etc/drbd.conf

global { 
  usage-count yes; 
}
common {
  protocol C;
}
resource r0 {
  on master {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   192.168.0.1:7789;
    meta-disk internal;
  }
  on slave {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   10.1.1.32:7789;
    meta-disk internal;
  }
}
			

72.1.4. Starting

Each of the following steps must be completed on both nodes.

neo@master:~$ sudo drbdadm create-md r0	
neo@master:~$ sudo drbdadm attach r0
neo@master:~$ sudo drbdadm connect r0
neo@master:~$ sudo drbdadm -- --overwrite-data-of-peer primary r0

neo@slave:~$ sudo drbdadm create-md r0		
neo@slave:~$ sudo drbdadm attach r0
neo@slave:~$ sudo drbdadm connect r0

			

master

			
neo@master:~$ sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

slave

			
neo@slave:~# sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

status

neo@master:~$ cat /proc/drbd
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43
 0: cs:StandAlone st:Primary/Unknown ds:UpToDate/DUnknown   r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
        resync: used:0/31 hits:0 misses:0 starving:0 dirty:0 changed:0
        act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0
 1: cs:Connected st:Secondary/Secondary ds:Diskless/Inconsistent C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
			
			

72.1.5. Using

master

neo@master:~$ sudo drbdadm primary all		
neo@master:~$ sudo mkfs.reiserfs /dev/drbd0
neo@master:~$ sudo mkdir /mnt/drbd0
neo@master:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@master:~$ sudo touch /mnt/drbd0/helloworld.tmp
neo@master:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             7.6G  1.3G  6.0G  18% /
varrun                125M  216K  125M   1% /var/run
varlock               125M  8.0K  125M   1% /var/lock
udev                  125M   60K  125M   1% /dev
devshm                125M     0  125M   0% /dev/shm
/dev/drbd0            2.0G   33M  2.0G   2% /mnt/drbd0
neo@master:~$ sudo dd if=/dev/zero of=/mnt/drbd0/tempfile1.tmp bs=104857600 count=1
1+0 records in
1+0 records out
104857600 bytes (105 MB) copied, 0.564911 s, 186 MB/s
neo@master:~$ sudo umount /mnt/drbd0/
neo@master:~$ sudo drbdadm secondary all
			

slave

neo@slave:~$ sudo drbdadm primary all			
neo@slave:~$ sudo mkdir /mnt/drbd0
neo@slave:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@slave:~$ ls /mnt/drbd0/
helloworld.tmp  tempfile1.tmp