目錄
過程 17.1. Hypertable standalone 安裝過程
安裝 Hypertable 軟件包
# cd /usr/local/src/ # wget http://cdn.hypertable.com/packages/0.9.7.0/hypertable-0.9.7.0-linux-x86_64.rpm
安裝 Hypertable， 我個人比較喜歡用yum localinstall他會解決軟件之間的依賴關係
# yum localinstall hypertable-0.9.7.0-linux-x86_64.rpm
相關的軟件會自動安裝
Dependencies Resolved ==================================================================================================================== Package Arch Version Repository Size ==================================================================================================================== Installing: hypertable x86_64 0.9.7.0-1 /hypertable-0.9.7.0-linux-x86_64 288 M Installing for dependencies: mailcap noarch 2.1.31-2.el6 base 27 k perl-Bit-Vector x86_64 7.1-2.el6 base 169 k perl-Carp-Clan noarch 6.03-2.el6 base 25 k perl-Compress-Raw-Zlib x86_64 1:2.020-127.el6 base 68 k perl-Compress-Zlib x86_64 2.020-127.el6 base 43 k perl-HTML-Parser x86_64 3.64-2.el6 base 109 k perl-HTML-Tagset noarch 3.20-4.el6 base 17 k perl-IO-Compress-Base x86_64 2.020-127.el6 base 67 k perl-IO-Compress-Zlib x86_64 2.020-127.el6 base 134 k perl-IO-String noarch 1.08-9.el6 base 15 k perl-URI noarch 1.40-2.el6 base 117 k perl-libwww-perl noarch 5.833-2.el6 base 387 k Transaction Summary ==================================================================================================================== Install 13 Package(s)
Hypertable 預設安裝在 /opt/hypertable/0.9.7.0
備份配置檔案，
# cd /opt/hypertable/0.9.7.0/conf # cp hypertable.cfg hypertable.cfg.original
FHS-IZE 安裝
# bin/fhsize.sh Setting up /var/opt/hypertable Setting up /etc/opt/hypertable fshize /opt/hypertable/0.9.7.0: success
設計 "CURRENT" 連接
# cd /opt/hypertable # ln -s 0.9.7.0 current
安裝 notification-hook.sh 腳本.
# cp conf/notification-hook.tmpl conf/notification-hook.sh # chmod o+x conf/notification-hook.sh
測試 notification-hook.sh腳本 .
/opt/hypertable/current/conf/notification-hook.sh "Test Message" "This is a test."
啟動 hypertable
# /opt/hypertable/current/bin/start-all-servers.sh local DFS broker: available file descriptors: 1024 Started DFS Broker (local) Started Hyperspace Started Hypertable.Master /proc/sys/vm/swappiness = 60 Started Hypertable.RangeServer Started ThriftBroker
# /opt/hypertable/current/bin/ht shell Welcome to the hypertable command interpreter. For information about Hypertable, visit http://hypertable.com Type 'help' for a list of commands, or 'help shell' for a list of shell meta commands. hypertable>
測試安裝是否有效
# /opt/hypertable/current/bin/ht shell Welcome to the hypertable command interpreter. For information about Hypertable, visit http://hypertable.com Type 'help' for a list of commands, or 'help shell' for a list of shell meta commands. hypertable> help USE ................ Sets the current namespace CREATE NAMESPACE ... Creates a new namespace DROP NAMESPACE ..... Removes a namespace EXISTS TABLE ....... Check if table exists CREATE TABLE ....... Creates a table DELETE ............. Deletes all or part of a row from a table DESCRIBE TABLE ..... Displays a table's schema DROP TABLE ......... Removes a table RENAME TABLE ....... Renames a table DUMP TABLE ......... Create efficient backup file ALTER TABLE ........ Add/remove column family from existing table INSERT ............. Inserts data into a table LOAD DATA INFILE ... Loads data from a TSV input file into a table SELECT ............. Selects (and display) cells from a table SHOW CREATE TABLE .. Displays CREATE TABLE command used to create table SHOW TABLES ........ Displays only the list of tables in the current namespace GET LISTING ........ Displays the list of tables and namespace in the current namespace Statements must be terminated with ';'. For more information on a specific statement, type 'help <statement>', where <statement> is from the preceding list. hypertable>quit
停止 hypertable
運行下列命令停止 Hypertable
$ /opt/hypertable/current/bin/stop-servers.sh
過程 17.2. Hypertable on HDFS
創建工作目錄
$ hadoop fs -mkdir /hypertable $ hadoop fs -chmod 777 /hypertable
安裝 Java 運行環境
yum install java-1.7.0-openjdk yum localinstall http://ftp.cuhk.edu.hk/pub/packages/apache.org/hadoop/common/hadoop-1.1.2/hadoop-1.1.2-1.x86_64.rpm
修改 jrun bug
cp /opt/hypertable/current/bin/jrun /opt/hypertable/current/bin/jrun.old
vim /opt/hypertable/current/bin/jrun
#HT_JAR=`ls -1 /opt/hypertable/doug/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'`
HT_JAR=`ls -1 /opt/hypertable/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'`
export JAVA_HOME=/usr export HADOOP_HOME=/usr export HYPERTABLE_HOME=/opt/hypertable/current
hypertable.cfg
# cat conf/hypertable.cfg # # hypertable.cfg # # HDFS Broker #HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf HdfsBroker.Hadoop.ConfDir=/etc/hadoop # Ceph Broker CephBroker.MonAddr=192.168.6.2:6789 # Local Broker DfsBroker.Local.Root=fs/local # DFS Broker - for clients DfsBroker.Port=38030 # Hyperspace Hyperspace.Replica.Host=localhost Hyperspace.Replica.Port=38040 Hyperspace.Replica.Dir=hyperspace # Hypertable.Master #Hypertable.Master.Host=localhost Hypertable.Master.Port=38050 # Hypertable.RangeServer Hypertable.RangeServer.Port=38060 Hyperspace.KeepAlive.Interval=30000 Hyperspace.Lease.Interval=1000000 Hyperspace.GracePeriod=200000 # ThriftBroker ThriftBroker.Port=38080
Hadoop 配置檔案 /etc/hadoop/core-site.xml
# cat /etc/hadoop/core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://namenode.example.com:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/var/tmp/hadoop</value>
</property>
</configuration>
Hadoop 配置檔案 /etc/hadoop/hdfs-site.xml
# cat /etc/hadoop/hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.name.dir</name>
<value>/var/hadoop/name1</value>
<description> </description>
</property>
<property>
<name>dfs.data.dir</name>
<value>/var/hadoop/hdfs/data1</value>
<description> </description>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>
啟動 dfsbroker
# /opt/hypertable/current/bin/set-hadoop-distro.sh cdh4 Hypertable successfully configured for Hadoop cdh4
# /opt/hypertable/current/bin/start-dfsbroker.sh hadoop DFS broker: available file descriptors: 1024 Started DFS Broker (hadoop)
查看啟動日誌
# tail -f /opt/hypertable/current/log/DfsBroker.hadoop.log log4j:WARN No appenders could be found for logger (org.apache.hadoop.conf.Configuration). log4j:WARN Please initialize the log4j system properly. HdfsBroker.dfs.client.read.shortcircuit=false HdfsBroker.dfs.replication=2 HdfsBroker.Server.fs.default.name=hdfs://namenode.example.com:9000 Apr 23, 2013 6:43:18 PM org.hypertable.AsyncComm.IOHandler DeliverEvent INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Connection Established Apr 23, 2013 6:43:18 PM org.hypertable.DfsBroker.hadoop.ConnectionHandler handle INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Disconnect - COMM broken connection : Closing all open handles from /192.168.6.25:53556 Closed 0 input streams and 0 output streams for client connection /192.168.6.25:53556
修改 CephBroker.MonAddr 對應的伺服器與連接埠號即可
# cat hypertable.cfg # # hypertable.cfg # # HDFS Broker HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf # Ceph Broker CephBroker.MonAddr=192.168.6.2:6789 # Local Broker DfsBroker.Local.Root=fs/local # DFS Broker - for clients DfsBroker.Port=38030 # Hyperspace Hyperspace.Replica.Host=localhost Hyperspace.Replica.Port=38040 Hyperspace.Replica.Dir=hyperspace # Hypertable.Master Hypertable.Master.Port=38050 # Hypertable.RangeServer Hypertable.RangeServer.Port=38060 Hyperspace.KeepAlive.Interval=30000 Hyperspace.Lease.Interval=1000000 Hyperspace.GracePeriod=200000 # ThriftBroker ThriftBroker.Port=38080
啟動 dfsbroker
# /opt/hypertable/current/bin/start-dfsbroker.sh ceph
創建一個表
# echo "USE '/'; CREATE TABLE foo ( c1, c2 ); GET LISTING;" \ > | /opt/hypertable/current/bin/ht shell --batch foo sys (namespace) tmp (namespace)
插入一些數據
# echo "USE '/'; INSERT INTO foo VALUES('001', 'c1', 'very'), \
> ('000', 'c1', 'Hypertable'), ('001', 'c2', 'easy'), ('000', 'c2', 'is');" \
> | /opt/hypertable/current/bin/ht shell --batch
查詢數據
# echo "USE '/'; SELECT * FROM foo;" \ > | /opt/hypertable/current/bin/ht shell --batch 000 c1 Hypertable 000 c2 is 001 c1 very 001 c2 easy
如果你想清楚所有表運行下面命令
$ /opt/hypertable/current/bin/stop-servers.sh $ /opt/hypertable/current/bin/clean-database.sh