Home | Mirror | Search | 雜文 | ITEYE 博客 | OSChina 博客 | 51CTO 博客

第 66 章 Distributed File Systems

目錄

66.1. DRBD (Distributed Replicated Block Device)
66.1.1. disk and partition
66.1.2. Installation
66.1.3. configure
66.1.4. Starting
66.1.5. Using
66.2. Network Block Device protocol
66.2.1. nbd-server - Network Block Device protocol - server
66.2.2. nbd-client - Network Block Device protocol - client
66.3. GridFS
66.3.1. nginx-gridfs
66.3.2. lighttpd-gridfs
66.4. Moose File System
66.4.1. Master server installation
66.4.2. Backup server (metalogger) installation
66.4.3. Chunk servers installation
66.4.4. Users’ computers installation
66.4.5. Testing MFS
66.5. Hadoop - HDFS
66.5.1. 單機安裝
66.5.2. 分散式安裝
66.5.2.1. 準備工作
66.5.2.2. NameNode 配置名稱節點
66.5.2.3. DataNode 配置數據節點
66.5.2.4. Hadoop UI (WEB界面)
66.5.2.5. 測試Hadoop
66.5.3. 二進制包安裝
66.5.3.1. 安裝 JRE
66.5.3.2. 安裝 Hadoop
66.5.3.3. 創建用戶
66.5.4. FAQ
66.5.4.1. hadoop-1.1.2-1.x86_64.rpm 包含哪些檔案內容
66.6. Ceph
66.6.1. Installation on Ubuntu
66.6.2. Installation on CentOS
66.6.2.1. mon
66.6.2.2. mds
66.6.2.3. osd
66.6.2.4. client
66.6.2.5. RADOS Gateway
66.6.3. Block Devices
66.7. GlusterFS
66.7.1. glusterfs-server
66.7.2. glusterfs-client
66.7.3. Testing
66.7.4. RAID
66.7.4.1. Mirror
66.7.4.2. Strip
66.7.5. Filesystem Administration
66.7.6. CentOS 6.3
66.8. Lustre
66.9. MogileFS
66.10. Kosmos distributed file system (KFS)
66.11. Coda
66.12. OpenAFS
66.13. Shared disk cluster file system
66.13.1. GFS2
66.13.2. OCFS2
66.13.3. fam & imon

66.1. DRBD (Distributed Replicated Block Device)

Homepage: http://www.drbd.org/

實驗環境需要兩台電腦,如果你沒有,建議你使用VMware,並且為每一個虛擬機添加兩塊硬碟。

實驗環境

  1. master: 192.168.0.1 DRBD:/dev/sdb

  2. slave: 192.168.0.2 DRBD:/dev/sdb

66.1.1. disk and partition

Each of the following steps must be completed on both nodes

show all of disk and partition

neo@master:~$ sudo sfdisk -s
/dev/sda:   8388608
/dev/sdb:   2097152
total: 10485760 blocks
			

create a new partition on the disk /dev/sdb

$ sudo cfdisk /dev/sdb			
			

you must have extended partition

check partition

neo@master:~$ sudo fdisk -l

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000301bd

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         993     7976241   83  Linux
/dev/sda2             994        1044      409657+   5  Extended
/dev/sda5             994        1044      409626   82  Linux swap / Solaris

Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         261     2096451    5  Extended
/dev/sdb5               1         261     2096419+  83  Linux

			

format /dev/sdb1

neo@master:~$ sudo mkfs.ext3 /dev/sdb1
			

you also can using other file system

reiserfs

neo@master:~$ sudo mkfs.reiserfs /dev/sdb1		
			

I suggest you using reiserfs.

66.1.2. Installation

Each of the following steps must be completed on both nodes

search drbd8-utils package

neo@master:~$ apt-cache search drbd
drbd8-utils - RAID 1 over tcp/ip for Linux utilities
drbd0.7-module-source - RAID 1 over tcp/ip for Linux module source
drbd0.7-utils - RAID 1 over tcp/ip for Linux utilities
drbdlinks - Manages symlinks into a shared DRBD partition
			

installation

neo@master:~$ sudo apt-get install drbd8-utils			
			

to add modules from the Linux Kernel

neo@master:~$ sudo modprobe drbd
neo@master:~$ lsmod |grep drbd
drbd                  213000  0
cn                      9632  1 drbd
			
			

66.1.3. configure

Each of the following steps must be completed on both nodes

backup configure file

			
neo@master:~$ sudo cp /etc/drbd.conf /etc/drbd.conf.old
			
			

edit /etc/drbd.conf

global { 
  usage-count yes; 
}
common {
  protocol C;
}
resource r0 {
  on master {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   192.168.0.1:7789;
    meta-disk internal;
  }
  on slave {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   10.1.1.32:7789;
    meta-disk internal;
  }
}
			

66.1.4. Starting

Each of the following steps must be completed on both nodes.

neo@master:~$ sudo drbdadm create-md r0	
neo@master:~$ sudo drbdadm attach r0
neo@master:~$ sudo drbdadm connect r0
neo@master:~$ sudo drbdadm -- --overwrite-data-of-peer primary r0

neo@slave:~$ sudo drbdadm create-md r0		
neo@slave:~$ sudo drbdadm attach r0
neo@slave:~$ sudo drbdadm connect r0

			

master

			
neo@master:~$ sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

slave

			
neo@slave:~# sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

status

neo@master:~$ cat /proc/drbd
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43
 0: cs:StandAlone st:Primary/Unknown ds:UpToDate/DUnknown   r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
        resync: used:0/31 hits:0 misses:0 starving:0 dirty:0 changed:0
        act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0
 1: cs:Connected st:Secondary/Secondary ds:Diskless/Inconsistent C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
			
			

66.1.5. Using

master

neo@master:~$ sudo drbdadm primary all		
neo@master:~$ sudo mkfs.reiserfs /dev/drbd0
neo@master:~$ sudo mkdir /mnt/drbd0
neo@master:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@master:~$ sudo touch /mnt/drbd0/helloworld.tmp
neo@master:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             7.6G  1.3G  6.0G  18% /
varrun                125M  216K  125M   1% /var/run
varlock               125M  8.0K  125M   1% /var/lock
udev                  125M   60K  125M   1% /dev
devshm                125M     0  125M   0% /dev/shm
/dev/drbd0            2.0G   33M  2.0G   2% /mnt/drbd0
neo@master:~$ sudo dd if=/dev/zero of=/mnt/drbd0/tempfile1.tmp bs=104857600 count=1
1+0 records in
1+0 records out
104857600 bytes (105 MB) copied, 0.564911 s, 186 MB/s
neo@master:~$ sudo umount /mnt/drbd0/
neo@master:~$ sudo drbdadm secondary all
			

slave

neo@slave:~$ sudo drbdadm primary all			
neo@slave:~$ sudo mkdir /mnt/drbd0
neo@slave:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@slave:~$ ls /mnt/drbd0/
helloworld.tmp  tempfile1.tmp			
			
comments powered by Disqus