Netkiller Database 手札

PostgreSQL, MySQL, Oracle ...

Mr. Neo Chan, 陈景峰(BG7NYT)

中国广东省深圳市龙华新区民治街道溪山美地
518131
+86 13113668890
+86 755 29812080
<netkiller@msn.com>

文档初创于April 6, 2008

版权 © 2009, 2010, 2011, 2012, 2013 Netkiller(Neo Chan). All rights reserved.

版权声明

转载请与作者联系，转载时请务必标明文章原始出处和作者信息及本声明。
文档出处:
http://netkiller.github.io
http://netkiller.sourceforge.net

$Date: 2013-05-21 19:01:33 +0800 (Tue, 21 May 2013) $

我的系列文档

Netkiller Architect 手札 | Netkiller Developer 手札 | Netkiller PHP 手札 | Netkiller Python 手札 | Netkiller Testing 手札 | Netkiller Cryptography 手札
Netkiller Linux 手札 | Netkiller CentOS 手札 | Netkiller FreeBSD 手札 | Netkiller Security 手札 | Netkiller Version 手札 | Netkiller Web 手札
Netkiller Monitoring 手札 | Netkiller Storage 手札 | Netkiller Mail 手札 | Netkiller Shell 手札 | Netkiller Network 手札 | Netkiller Database 手札
Netkiller PostgreSQL 手札 | Netkiller MySQL 手札 | Netkiller NoSQL 手札 | Netkiller LDAP 手札 | Netkiller Cisco IOS 手札 | Netkiller H3C 手札
Netkiller Multimedia 手札 | Netkiller Docbook 手札 | Netkiller 开源软件 手札 |  |  |

您可以使用iBook阅读当前文档

QQ群：128659835 请注明“读者”

目录

自述
1. 系列文档
2. 作者简介
3. 写给读者
I. Database Relational
1. DB-Engines - DB-Engines Ranking
2. SysBench
3. SuperSmack
4. ETL (Extract-Transform-Load)
5. 数据迁移
5.1. Apache Sqoop
6. GreenSQL
7. Database design & E-R diagram
7.1. opensource database design tools
7.2. OpenSystemArchitect
7.3. SQL Power Architect
II. MySQL
8. MySQL Server
8.1. MySQL Installation
8.1.1. Installation by apt-get under debian/ubuntu
8.1.2. Installation by source code
8.1.3. MySQL binary distribution
8.1.4. mysql-5.5.21-debian6.0-i686.deb
8.1.5. CentOS 6.2 + MySQL 5.5.25 (RPM)
8.1.6. mysql-admin
8.2. my.cnf
8.2.1. bind-address
8.2.2. Configuring Database Character Encoding
8.2.3. max_connections
8.2.4. storage-engine
8.2.5. max_allowed_packet
8.2.6. skip-name-resolve
8.2.7. 禁用TCP/IP链接
8.2.8. timeout
8.2.9. 与复制有关的参数
8.2.10. 与 InnoDB 有关的配置项
8.2.11. Example for my.cnf
8.3. Replication
8.3.1. Master Slave
8.3.2. Master Master(主主)
8.3.3. 与复制有关的问题
8.4. MySQL Custer
8.4.1. Management node (MGM node)
8.4.2. Data node
8.4.3. SQL node
8.4.4. Starting
8.4.5. Shutdown
8.4.6. Testing
8.5. MySQL Proxy
8.5.1. Ubuntu
8.5.2. CentOS
8.6. Security
8.6.1. Firewall
8.6.2. 保护表字段
8.7. Monitoring
8.7.1. Analysis and Optimization
8.7.2. Munin
8.7.3. Cacti
8.7.4. Monitoring MySQL with SNMP
9. Client and Utility Programs
9.1. mysql - the MySQL command-line tool
9.1.1. ~/.my.cnf
9.1.2. 屏幕输出到文件
9.1.3. 终端编码
9.1.4. Unix Socket
9.1.5. 重定向巧用
9.2. mysqldump - a database backup program
9.2.1. 备份数据库，无结构，只有数据
9.2.2. 备份数据库结构（不备份数据）
9.2.3. 使用完整的insert插入数据
9.2.4. --extended-insert / --skip-extended-insert
9.2.5. --where
9.3. mysqladmin - client for administering a MySQL server
9.3.1. reload
9.3.2. 更改密码
9.3.3. status
9.3.4. process list
9.4. myisamchk — MyISAM Table-Maintenance Utility
9.5. mysqlcheck — A Table Maintenance and Repair Program
9.6. mysqlslap - load emulation client
9.7. mysqldumpslow - Parse and summarize the MySQL slow query log.
9.8. mysql log
10. Backup and Recovery
10.1. Import / Export
10.1.1. Export(Backup)
10.1.2. Import(Recovery)
10.1.3. xml
10.1.4. 备份表数据
10.1.5. source
10.1.6. 使用 mysqlhotcopy 备份 MyISAM 引擎的数据库
10.1.7. AutoMySQLBackup
10.1.8. xtrabackup - Open source backup tool for InnoDB and XtraDB.
10.2. Snapshot Backup
10.2.1. LVM Snapshot
10.2.2. Btrfs Snapshot
11. Database Administration
11.1. User Account Management
11.1.1. Create User
11.1.2. Drop User
11.1.3. Rename User
11.1.4. SET PASSWORD
11.1.5. Reset root password
11.2. Access Privilege System
11.2.1. SHOW GRANTS
11.2.2. show privileges
11.2.3. Grant privileges
11.2.4. Revoke privileges
11.2.5. Show Privileges
11.2.6. MAX_QUERIES_PER_HOUR/MAX_UPDATES_PER_HOUR
11.3. Maintenance 数据库维护
11.3.1. CHECK 检查表
11.3.2. ANALYZE 分析表
11.3.3. CHECKSUM
11.3.4. OPTIMIZE 优化表
11.3.5. REPAIR 修复
11.4. SHOW COMMAND
11.4.1. 查看版本
11.4.2. status
11.4.3. show processlist
11.4.4. variables
11.4.5. binary
12. DDL - Data Definition Language
12.1. Database
12.1.1. create
12.1.2. drop
12.1.3. Alter
12.1.4. Rename
12.1.5. CHARACTER
12.1.6. show create database
12.2. Table
12.2.1. create table ... select
12.2.2. modifiy table
12.2.3. TEMPORARY Table
12.2.4. Collate
12.2.5. CHARACTER
12.2.6. DEFAULT
12.2.7. KEY
12.2.8. COMMENT
12.3. Index
12.3.1. SHOW INDEX
12.3.2. CREATE INDEX
12.3.3. DROP INDEX
12.3.4. rebuild
12.4. 外键(Foreign Key)
12.4.1. FOREIGN KEY (RESTRICT)
12.5. View
12.6. Trigger
12.6.1. create trigger
12.6.2. drop trigger
12.6.3. show triggers
12.6.4. EXAMPLE
12.7. EVENT
12.7.1. 启用 EVENT
12.7.2. 创建 EVENT
12.7.3. 禁用/启用
12.7.4. show events
12.8. Partitioning
12.8.1. RANGE
12.8.2. LIST
12.8.3. HASH
12.8.4. KEY分区
12.8.5. 分区管理
12.8.6. EXPLAIN PARTITIONS
12.8.7. SHOW CREATE TABLE
12.8.8. INFORMATION_SCHEMA.partitions 表
12.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
12.9. Engine
12.9.1. FEDERATED
12.9.2. BLACKHOLE
13. DML (Data Manipulation Language)
13.1. INSERT
13.1.1. INSERT INTO ... SELECT
14. SQL Statement Syntax
14.1. DISTINCT
14.2. group by
14.3. replace
14.4. OUTFILE/LOAD DATA INFILE
14.4.1. Export data to CSV from MySQL
14.5. SQL 92
14.6. Functions and Operators
14.6.1. COUNT
14.6.2. UUID()
14.6.3. String
14.6.4. Date and Time
14.7. 事务处理和锁定语句
14.7.1. 排他锁
14.7.2. 表的加锁与解锁
14.7.3. 事务所用到的表
14.8. MySQL 专有命令
14.8.1. SQL_NO_CACHE
15. Optimization
15.1. Buffering and Caching
15.1.1. Query Cache SELECT Options
15.2. where 优化
16. MySQL Connectors
16.1. JDBC
16.2. ODBC
16.3. MySQL native driver for PHP - mysqlnd
16.4. python-mysqldb
17. MySQL GUI/Web Manager
17.1. HeidiSQL
17.2. Toad for MySQL Freeware
17.3. phpMyAdmin - MySQL web administration tool
17.4. Maatkit Essential command-line utilities for MySQL
18. Miscellaneous
18.1. Multi-Master Replication Manager for MySQL
18.2. HandlerSocket
18.3. MariaDB
18.4. Maatkit
19. FAQ
19.1. 数据库内容替换
19.2. 查看错误代码
19.2.1. ERROR 1153 (08S01) at line 3168: Got a packet bigger than 'max_allowed_packet' bytes
19.2.2. ERROR 1129 (00000): Host 'XXXXXX' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
19.3. 临时表是否需要建索引
19.4. Kill 脚本
III. PostgreSQL
20. PostgreSQL 安装
20.1. Ubuntu 12.04.1 LTS 安装
20.2. YUM 默认源安装
20.3. PostgreSQL YUM 源安装
20.4. PostgreSQL 配置
20.4.1. postgresql.conf
20.4.2. pg_hba.conf
20.5. 创建dba用户
21. Administration
21.1. User 用户管理
21.1.1. SUPERUSER
21.1.2. 删除用户
21.2. Database
21.2.1. 删除数据库
21.3. Table
21.4. Backup / Restore
21.4.1. Backup
21.4.2. Restore
21.4.3. import
21.4.4. PostgreSQL for windows
21.4.5. 大型数据库备份
22. PostgreSQL 系统表
22.1. pg_available_extensions
22.2. pg_shadow
23. DDL - Data Definition Language
23.1. Database
23.1.1. create
23.1.2. drop
23.1.3. Alter
23.1.4. Rename
23.1.5. CHARACTER
23.1.6. show create database
23.2. Table
23.2.1. create table ... select
23.2.2. modifiy table
23.2.3. TEMPORARY Table
23.2.4. Collate
23.2.5. CHARACTER
23.2.6. DEFAULT
23.2.7. KEY
23.2.8. COMMENT
23.3. Index
23.3.1. SHOW INDEX
23.3.2. CREATE INDEX
23.3.3. DROP INDEX
23.3.4. rebuild
23.4. 外键(Foreign Key)
23.4.1. FOREIGN KEY (RESTRICT)
23.5. View
23.6. Trigger
23.6.1. create trigger
23.6.2. drop trigger
23.6.3. show triggers
23.6.4. EXAMPLE
23.7. EVENT
23.7.1. 启用 EVENT
23.7.2. 创建 EVENT
23.7.3. 禁用/启用
23.7.4. show events
23.8. Partitioning
23.8.1. RANGE
23.8.2. LIST
23.8.3. HASH
23.8.4. KEY分区
23.8.5. 分区管理
23.8.6. EXPLAIN PARTITIONS
23.8.7. SHOW CREATE TABLE
23.8.8. INFORMATION_SCHEMA.partitions 表
23.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
23.9. Engine
23.9.1. FEDERATED
23.9.2. BLACKHOLE
24. DML (Data Manipulation Language)
24.1. INSERT
24.1.1. INSERT INTO ... SELECT
25. SQL Statement Syntax
25.1. DISTINCT
25.2. group by
25.3. replace
25.4. OUTFILE/LOAD DATA INFILE
25.4.1. Export data to CSV from MySQL
25.5. SQL 92
25.6. Functions and Operators
25.6.1. COUNT
25.6.2. UUID()
25.6.3. String
25.6.4. Date and Time
25.7. 事务处理和锁定语句
25.7.1. 排他锁
25.7.2. 表的加锁与解锁
25.7.3. 事务所用到的表
25.8. MySQL 专有命令
25.8.1. SQL_NO_CACHE
26. PostgreSQL GUI
26.1. pgAdmin III
26.2. phpPgAdmin
26.3. Monitoring
26.3.1. PgBadger
26.4. pgModeler - PostgreSQL Database Modeler
27. Barman Backup & recovery for PostgreSQL
28. pgbouncer - lightweight connection pooler for PostgreSQL
28.1. 安装 pgbouncer
28.1.1. Ubuntu
28.1.2. CentOS
28.2. 配置 pgbouncer
29. Foreign data wrappers
30. FAQ
IV. Oracle
31. Installing Oracle Database
31.1. Installing Oracle Database 10g Release 2 on Linux x86
31.2. 11gR2
31.3. Installing Oracle Client
31.4. Silence Install - Database
31.4.1. Response File
31.4.2. OS 配置脚本
31.4.3. 运行 runInstaller
31.5. Silence Install - Client
32. Manager
32.1. listener.ora
32.1.1. TNS 配置
32.2. Account
32.2.1. show user
32.2.2. DEFAULT_TABLESPACE
32.2.3. unlock/lock
32.3. 显示表
32.4. oracle 817 script
32.5. Script for automatic startup on boot
32.6. Run level shell script to start Oracle 10g services on RedHat Enterprise Linux (RHAS 4)
33. Oracle Gui
33.1. Create instance
33.2. Oracle Net Configuration Assistant
33.3. Oracle Enterprise Manager
33.4. Other GUI - phpOraAdmin
34. CLI
34.1. SQL*Plus
34.1.1. startup/shutdown
34.1.2. conn
34.1.3. parameter
34.1.4. $ORACLE_HOME/sqlplus/admin/glogin.sql
34.1.5. @运行SQL
34.2. lsnrctl
34.3. RMAN
34.3.1. 数据库模式
34.3.2. 完全备份
34.3.3. 增量备份
34.3.4. 恢复数据库
34.3.5. 是用tar打包rman文件
V. NoSQL
35. MongoDB
35.1. Quickstart
35.1.1. 二进制tar包安装
35.1.2. Ubuntu MongoDB
35.1.3. CentOS MongoDB
35.1.4. MongoDB + Hadoop
35.2. Security and Authentication
35.2.1. 超级管理员
35.2.2. 数据库用户
35.2.3. 删除用户
35.3. Replication
35.4. Drivers
35.4.1. Using MongoDB in PHP
35.5. mongo 命令行
35.5.1. help
35.5.2. show 查看命令
35.5.3. 切换数据库
35.5.4. 索引管理
35.5.5. 记录管理
35.5.6. 管道操作
35.6. Map-Reduce
35.6.1. 使用 Map-Reduce 统计Web 服务器 access.log 日志文件
35.7. Mongo Admin UI
35.7.1. RockMongo
35.7.2. MongoVUE
36. Redis
36.1. Redis 安装
36.1.1. CentOS
36.1.2. Ubuntu
36.1.3. Test Redis
36.2. /etc/redis.conf
36.2.1. 密码认证
36.3. redis-cli - Command-line client to redis-server
36.3.1. info
36.3.2. save/bgsave/lastsave
36.3.3. Pub/Sub 订阅与发布
36.4. phpRedisAdmin
37. Cassandra
37.1. Getting Started
37.1.1. Downloading and Installation
37.1.2. Running Cassandra
37.1.3. cli tool
37.1.4. Testing Cassandra
37.2. Configure Cassandra
37.2.1. Envionment variables
37.2.2. log4j.properties
37.2.3. storage-conf.xml
37.3. Keyspace
37.3.1. Schema
37.3.2. Keyspace example
37.4. Cluster
37.4.1. Running a cluster
37.4.2. Running a single node
37.4.3. nodetool
38. Hypertable
38.1. Hypertable 安装
38.1.1. Hypertable standalone 单机安装
38.1.2. Hypertable on HDFS(hadoop) 安装
38.1.3. MapR
38.1.4. Ceph
38.1.5. 检验安装
38.2. Code examples
38.2.1. PHP
38.3. HQL
38.3.1. namespace 命名空间管理
38.3.2. Table 表
38.4. FAQ
38.4.1. 切换 DFS Broker
39. Membase
40. TokyoCabinet/Tyrant
41. Flare
42. Voldemort
43. LevelDB
44. CouchDB
45. RethinkDB
46. HyperDex
VI. Search Engine
47. Solr
47.1. Embedded Jetty
47.2. Jetty
47.3. Tomcat
47.4. solr-php-client
47.5. multicore
47.6. 中文分词
47.6.1. ChineseTokenizerFactory
47.6.2. CJK
47.6.3. mmseg4j
47.6.4. 中文分词“庖丁解牛” Paoding Analysis
48. Nutch
49. Sphinx
50. Lucene
51. MG4J
52. PhpDig
53. Mahout
54. elasticsearch

范例清单

8.1. my.cnf
12.1. BEFORE/AFTER
12.2. uuid()
15.1. SQL_CACHE 测试
23.1. BEFORE/AFTER
23.2. uuid()
28.1. /etc/pgbouncer/pgbouncer.ini
34.1. full backup
35.1. MongoDB Test
35.2. Using MongoDB in PHP
37.1. Twitter
37.2. Twissandra
47.1. /etc/profile.d/java.sh


自述

目录

1. 系列文档
2. 作者简介
3. 写给读者

目前技术书籍的价格一路飙升，动则80，100，少则50，60. 技术书籍有时效性，随着技术的革新或淘汰，大批书记成为废纸垃圾。并且这些书技术内容雷同，相互抄袭，质量越来越差，甚至里面给出的例子错误百出，只能购买影印版，或者翻译的版本。

在这种背景下我便萌生了自己写书的想法，资料主要来源是我的笔记与例子。我并不想出版，只为分享，所有我制作了基于CC License 发行的系列电子书。

本书注重例子，少理论，只要你对这例子一步一步操作，就会成功，会让你有成就感并能坚持学下去，因为很多人遇到障碍就会放弃，其实我就是这种人，只要让他看到希望，就能坚持下去。

为什么写这篇文章

有很多想法,工作中也用不到所以未能实现，所以想写出来,和大家分享.有一点写一点,写得也不好,只要能看懂就行,就当学习笔记了.

开始零零碎碎写过一些文档，也向维基百科供过稿，但维基经常被ZF封锁，后来发现sf.net可以提供主机存放文档，便做了迁移。并开始了我的写作生涯。

这篇文档是作者10年来对工作的总结,是作者一点一滴的积累起来的，有些笔记已经丢失，所以并不完整。

因为工作太忙整理比较缓慢。目前的工作涉及面比较窄所以新文档比较少。

我现在花在技术上的时间越来越少，兴趣转向摄影，无线电。也想写写摄影方面的心得体会。

写作动力:

曾经在网上看到外国开源界对中国的评价，中国人对开源索取无度，但贡献却微乎其微.这句话一直记在我心中，发誓要为中国开源事业做我仅有的一点微薄贡献

另外写文档也是知识积累，还可以增加在圈内的影响力.

人跟动物的不同,就是人类可以把自己学习的经验教给下一代人.下一代在上一代的基础上再创新,不断积累才有今天.

所以我把自己的经验写出来,可以让经验传承

没有内容的章节:

目前我自己一人维护所有文档，写作时间有限，当我发现一个好主题就会加入到文档中，待我有时间再完善章节，所以你会发现很多章节是空无内容的.

文档目前几乎是流水帐试的写作，维护量很大，先将就着看吧.

我想到哪写到哪,你会发现文章没一个中心,今天这里写点,明天跳过本章写其它的.
文中例子绝对多,对喜欢复制然后粘贴朋友很有用,不用动手写,也省时间.
理论的东西,网上大把,我这里就不写了,需要可以去网上查.
我爱写错别字,还有一些是打错的,如果发现请指正.
文中大部分试验是在Debian/Ubuntu/Redhat AS上完成.

1. 系列文档

下面是我多年积累下来的经验总结，整理成文档供大家参考:

HTML 格式

http://netkiller.github.com/home/technology.html

http://netkiller.sourceforge.net/home/technology.html

PDF 格式

Download PDF Document 下载PDF文档1

Download PDF Document 下载PDF文档2

EPUB 格式

http://netkiller.sourceforge.net/technology.html

通过 GIT 镜像整个网站

https://github.com/netkiller/netkiller.github.com.git

$ git clone https://github.com/netkiller/netkiller.github.com.git

其他方式

获得光盘介质，RPM包，DEB包，如有特别需要，请联系我


2. 作者简介

陈景峰 (ㄔㄣ ㄐ丨ㄥ ㄈㄥ)

Nickname：netkiller | English name: Neo chen | Nippon name: ちんけいほう (音訳) | Korean name: 천징봉 | Thailand name: ภูมิภาพภูเขา

Callsign: BG7NYT | QTH: ZONE CQ24 ITU44 ShenZhen, China

程序猿，攻城狮，挨踢民工， UNIX like Evangelist, 业余无线电爱好者（呼号：BG7NYT）,户外运动，山地骑行以及摄影爱好者。

《Netkiller 系列 手札》的作者

1981年1月19日(庚申年腊月十四)出生于黑龙江省青冈县建设乡双富大队第一小队

1989年9岁随父母迁居至黑龙江省伊春市

1996年第一次接触UNIX操作系统，BSD UNIX, Microsoft Xinux(盖茨亲自写的微软Unix，知道的人不多)

1997年第自学BASIC语言，Turbo C语言，苦于没有电脑，后来学校建了微机室才第一次使用QBASIC(DOS 6.22 自带命令)，那个年代只能通过软盘拷贝转播Trubo C编译器始终没有搞到，

1997年第一次上Internet网速只有9600Bps, 当时全国兴起各种信息港域名格式是www.xxxx.info.net, 访问的第一个网站是NASA下载了很多火星探路者拍回的照片，还有“淞沪”sohu的前身

1998~2000年在哈尔宾学习计算机，充足的上机时间，但老师让我们练打字（明伦五笔/WT）打字不超过80个/每分钟还要强化训练，不过这个给我的键盘功夫打了好底。

1999年学校的电脑终于安装了光驱，在一张工具盘上终于找到了Turbo C与Quick Basic编译器，当时对VGA图形编程非常感兴趣，通过INT33中断控制鼠标，使用绘图函数模仿windows界面。还有操作 UCDOS 中文字库，绘制矢量与点阵字体。

2000年沉迷于Windows NT与Back Office各种技术，神马主域控制器，DHCP，WINS，IIS，域名服务器，Exchange邮件服务器，MS Proxy, NetMeeting...以及ASP+MS SQL开发

职业生涯

2001年来深圳进城打工,成为一名外来务工者. 在一个4人公司做PHP开发，当时PHP的版本是2。0, 开始使用Linux Redhat 6.2.当时很多门户网站都是用FreeBSD,但很难搞到安装盘，在网易社区认识了一个网友,从广州给我寄了一张光盘，FreeBSD 3.2

2002年我发现不能埋头苦干,还要学会"做人".同住辗转广州工作了半年。

2003年这年最惨,公司拖欠工资16000元,打过两次官司2005才付清.

2004年开始加入分布式计算团队,目前成绩

2004-10月开始玩户外和摄影

2005-6月成为中国无线电运动协会会员,呼号BG7NYT,进了一部Yaesu FT-60R手台。

2006年单身生活了这么多年,终于找到归宿.

2007物价上涨,金融危机，休息了4个月（其实是找不到工作）,关外很难上439.460中继，搞了一台Yaesu FT-7800.

2008终于找到英文学习方法，， 《Netkiller Developer 手札》，《Netkiller Document 手札》

2008-8-8 08:08:08 结婚,后全家迁居湖南省常德市

2009《Netkiller Database 手札》,年底拿到C1驾照

2010对电子打击乐产生兴趣，计划学习爵士鼓

2011 平凡的一年，户外运动停止，电台很少开，中继很少上，摄影主要是拍女儿与家人，年末买了一辆山地车

2012 对油笔画产生了兴趣，活动基本是骑行银湖山绿道，

2013开始学习民谣吉他，同时对电吉他也极有兴趣；最终都放弃了。这一年深圳开始推数字中继2013-7-6日入手Motorola MOTOTRBO XIR P8668

职业生涯路上继续打怪升级

联系作者

http://netkiller.github.io/

http://netkiller.sourceforge.net/

Mobile: +86 13113668890

Tel: +86 755 2981-2080

Callsign: BG7NYT QTH: Shenzhen, China

注：请不要问我安装问题！

Email: netkiller@msn.com

MSN: netkiller@msn.com

ICQ: 101888222

QQ: 13721218

Yahoo/AIM/GTalk: bg7nyt

IRC: irc://irc.freenode.net /nickname netkiller or neo

CQ CQ CQ DE BG7NYT

写给火腿:

欢迎无线电爱好者和我QSO,我的QTH在深圳宝安区龙华镇溪山美地12B7CD,设备YAESU FT-50R,FT-60R,FT-7800 144-430双段机, ,拉杆天线/GP天线 Nagoya MAG-79EL-3W/Yagi

如果这篇文章对你有所帮助,请寄给我一张QSL卡片,qrz.cn or qrz.com or hamcall.net

Personal Amateur Radiostations of P.R.China

ZONE CQ24 ITU44 ShenZhen, China

Best Regards, VY 73! OP. BG7NYT

Blogger

Twitter: http://twitter.com/bg7nyt

Digg: http://www.digg.com/netkiller

Facebook: https://www.facebook.com/bg7nyt

LinkedIn: http://cn.linkedin.com/in/netkiller

Flickr: http://www.flickr.com/photos/bg7nyt/

Disqus: http://disqus.com/netkiller/

solidot: http://solidot.org/~netkiller/

ITEYE: http://netkiller-github-com.iteye.com

51CTO: http://bg7nyt.blog.51cto.com/|http://g.51cto.com/netkiller

pengyou: http://www.pengyou.com/netkiller

weibo: http://weibo.com/bg7nyt

163 Blog: http://t.163.com/netkiller

Mailing List

http://groups.google.com/group/netkiller

Send email to this group: netkiller@googlegroups.com

RSS http://groups.google.com/group/netkiller/feed/rss_v2_0_topics.xml?num=50


3. 写给读者

写给读者

欢迎提出宝贵的建议,如有问题请到 邮件列表 讨论

至读者：
我不知道什么时候，我不再更新文档或者退出IT行业去从事其他工作，我必须给这些文档找一个归宿，让他能持续更新下去。
我想捐赠给某些基金会继续运转，或者建立一个团队维护它。
我用了10年时间坚持不停地写作，持续更新，才有今天你看到的《Netkiller 手扎》系列文档，在中国能坚持10年，同时没有任何收益的技术类文档，是非常不容易的。
有很多时候想放弃，看到外国读者的支持与国内社区的影响，我坚持了下来。
中国开源事业需要各位参与，不要成为局外人，不要让外国人说：中国对开源索取无度，贡献却微乎其微。
我们参与内核的开发还比较遥远，但是进个人能力，写一些文档还是可能的。


部分 I. Database Relational

目录

1. DB-Engines - DB-Engines Ranking
2. SysBench
3. SuperSmack
4. ETL (Extract-Transform-Load)
5. 数据迁移
5.1. Apache Sqoop
6. GreenSQL
7. Database design & E-R diagram
7.1. opensource database design tools
7.2. OpenSystemArchitect
7.3. SQL Power Architect
What are the difference between DDL, DML and DCL commands?
DDL is Data Definition Language statements. Some examples:
CREATE - to create objects in the database
ALTER - alters the structure of the database
DROP - delete objects from the database
TRUNCATE - remove all records from a table, including all spaces allocated for

the records are removed
COMMENT - add comments to the data dictionary
GRANT - gives user's access privileges to database
REVOKE - withdraw access privileges given with the GRANT command

DML is Data Manipulation Language statements. Some examples:
SELECT - retrieve data from the a database
INSERT - insert data into a table
UPDATE - updates existing data within a table
DELETE - deletes all records from a table, the space for the records remain
CALL - call a PL/SQL or Java subprogram
EXPLAIN PLAN - explain access path to data
LOCK TABLE - control concurrency

DCL is Data Control Language statements. Some examples:
COMMIT - save work done
SAVEPOINT - identify a point in a transaction to which you can later roll back
ROLLBACK - restore database to original since the last COMMIT
SET TRANSACTION - Change transaction options like what rollback segment to use


第 1 章 DB-Engines - DB-Engines Ranking

http://db-engines.com/en/ranking


第 2 章 SysBench


第 3 章 SuperSmack


第 4 章 ETL (Extract-Transform-Load)

ETL或ELT （extract）、转置（transform）、加载（load）

http://zh.wikipedia.org/wiki/ETL

Kettle http://www.ketl.org

http://www.cloveretl.org/

http://www.xaware.org/

Apatar http://www.apatar.org/

http://www.enhydra.org/


第 5 章 数据迁移

data migration

目录

5.1. Apache Sqoop

5.1. Apache Sqoop

http://sqoop.apache.org/

export SQOOP_HOME=/srv/sqoop/
export PATH=$SQOOP_HOME/bin:$PATH

MYSQL转HDFS-示例

./sqoop import --connect jdbc:mysql://192.168.0.1:3306/mydb --username root --password mypass --table mytable -m 1 --target-dir /mysql/yourtable/$today

HDFS转MYSQ-示例

./sqoop export --connect jdbc:mysql://192.168.0.1:3306/mydb --username root --password mypass --table mytable --fields-terminated-by ',' --export-dir /mysql/yourtable/$today


第 6 章 GreenSQL


第 7 章 Database design & E-R diagram

目录

7.1. opensource database design tools
7.2. OpenSystemArchitect
7.3. SQL Power Architect

7.1. opensource database design tools

DBDesigner 4 - http://www.fabforce.net/dbdesigner4/


7.2. OpenSystemArchitect

http://www.codebydesign.com


7.3. SQL Power Architect

http://www.sqlpower.ca/


部分 II. MySQL

目录

8. MySQL Server
8.1. MySQL Installation
8.1.1. Installation by apt-get under debian/ubuntu
8.1.2. Installation by source code
8.1.3. MySQL binary distribution
8.1.4. mysql-5.5.21-debian6.0-i686.deb
8.1.5. CentOS 6.2 + MySQL 5.5.25 (RPM)
8.1.6. mysql-admin
8.2. my.cnf
8.2.1. bind-address
8.2.2. Configuring Database Character Encoding
8.2.3. max_connections
8.2.4. storage-engine
8.2.5. max_allowed_packet
8.2.6. skip-name-resolve
8.2.7. 禁用TCP/IP链接
8.2.8. timeout
8.2.9. 与复制有关的参数
8.2.9.1. 定义 log-bin 文件名
8.2.9.2. expire-logs-days
8.2.10. 与 InnoDB 有关的配置项
8.2.11. Example for my.cnf
8.3. Replication
8.3.1. Master Slave
8.3.1.1. Master
8.3.1.2. Slave
8.3.1.3. Testing
8.3.1.4. 将现有数据库迁移到主从结构数据库
8.3.1.5. 主从复制安全问题
8.3.1.6. 主从不同步问题
8.3.2. Master Master(主主)
8.3.2.1. Master A
8.3.2.2. Master B
8.3.2.3. 将Master A 数据库 同步到 Master B 两端数据库内容保持一致
8.3.2.4. Master A - B 同步两端数据库
8.3.2.5. Master A 数据库解除只读权限
8.3.2.6. 查看主主的工作状态
8.3.3. 与复制有关的问题
8.3.3.1. mysql-bin 清理问题
8.3.3.2. Semisynchronous Replication
8.4. MySQL Custer
8.4.1. Management node (MGM node)
8.4.2. Data node
8.4.3. SQL node
8.4.4. Starting
8.4.5. Shutdown
8.4.6. Testing
8.5. MySQL Proxy
8.5.1. Ubuntu
8.5.2. CentOS
8.5.2.1. FAQ
8.6. Security
8.6.1. Firewall
8.6.2. 保护表字段
8.7. Monitoring
8.7.1. Analysis and Optimization
8.7.1.1. mytop - top like query monitor for MySQL
8.7.1.2. mtop - MySQL terminal based query monitor
8.7.1.3. innotop
8.7.1.4. mysqlreport
8.7.2. Munin
8.7.3. Cacti
8.7.4. Monitoring MySQL with SNMP
9. Client and Utility Programs
9.1. mysql - the MySQL command-line tool
9.1.1. ~/.my.cnf
9.1.2. 屏幕输出到文件
9.1.3. 终端编码
9.1.4. Unix Socket
9.1.5. 重定向巧用
9.2. mysqldump - a database backup program
9.2.1. 备份数据库，无结构，只有数据
9.2.2. 备份数据库结构（不备份数据）
9.2.3. 使用完整的insert插入数据
9.2.4. --extended-insert / --skip-extended-insert
9.2.5. --where
9.3. mysqladmin - client for administering a MySQL server
9.3.1. reload
9.3.2. 更改密码
9.3.3. status
9.3.4. process list
9.4. myisamchk — MyISAM Table-Maintenance Utility
9.5. mysqlcheck — A Table Maintenance and Repair Program
9.6. mysqlslap - load emulation client
9.7. mysqldumpslow - Parse and summarize the MySQL slow query log.
9.8. mysql log
10. Backup and Recovery
10.1. Import / Export
10.1.1. Export(Backup)
10.1.2. Import(Recovery)
10.1.3. xml
10.1.4. 备份表数据
10.1.5. source
10.1.6. 使用 mysqlhotcopy 备份 MyISAM 引擎的数据库
10.1.7. AutoMySQLBackup
10.1.8. xtrabackup - Open source backup tool for InnoDB and XtraDB.
10.1.8.1. Percona yum Repository
10.1.8.2. Creating an Incremental Backup
10.2. Snapshot Backup
10.2.1. LVM Snapshot
10.2.2. Btrfs Snapshot
11. Database Administration
11.1. User Account Management
11.1.1. Create User
11.1.2. Drop User
11.1.3. Rename User
11.1.4. SET PASSWORD
11.1.5. Reset root password
11.2. Access Privilege System
11.2.1. SHOW GRANTS
11.2.2. show privileges
11.2.3. Grant privileges
11.2.4. Revoke privileges
11.2.5. Show Privileges
11.2.6. MAX_QUERIES_PER_HOUR/MAX_UPDATES_PER_HOUR
11.3. Maintenance 数据库维护
11.3.1. CHECK 检查表
11.3.2. ANALYZE 分析表
11.3.3. CHECKSUM
11.3.4. OPTIMIZE 优化表
11.3.5. REPAIR 修复
11.4. SHOW COMMAND
11.4.1. 查看版本
11.4.2. status
11.4.2.1. show status
11.4.2.2. show master status
11.4.2.3. show slave status
11.4.2.4. show plugins
11.4.3. show processlist
11.4.4. variables
11.4.4.1. wait_timeout
11.4.4.2. table_lock_wait_timeout
11.4.4.3. low_priority_updates
11.4.4.4. character_set
11.4.4.5. datadir
11.4.4.6. storage_engine
11.4.4.7. timeout
11.4.5. binary
12. DDL - Data Definition Language
12.1. Database
12.1.1. create
12.1.2. drop
12.1.3. Alter
12.1.4. Rename
12.1.5. CHARACTER
12.1.6. show create database
12.2. Table
12.2.1. create table ... select
12.2.2. modifiy table
12.2.3. TEMPORARY Table
12.2.4. Collate
12.2.5. CHARACTER
12.2.6. DEFAULT
12.2.7. KEY
12.2.7.1. PRIMARY KEY
12.2.8. COMMENT
12.3. Index
12.3.1. SHOW INDEX
12.3.2. CREATE INDEX
12.3.3. DROP INDEX
12.3.4. rebuild
12.4. 外键(Foreign Key)
12.4.1. FOREIGN KEY (RESTRICT)
12.5. View
12.6. Trigger
12.6.1. create trigger
12.6.2. drop trigger
12.6.3. show triggers
12.6.4. EXAMPLE
12.7. EVENT
12.7.1. 启用 EVENT
12.7.2. 创建 EVENT
12.7.3. 禁用/启用
12.7.4. show events
12.8. Partitioning
12.8.1. RANGE
12.8.2. LIST
12.8.3. HASH
12.8.3.1. LINEAR HASH
12.8.4. KEY分区
12.8.5. 分区管理
12.8.5.1. 新增分区
12.8.5.2. 删除分区
12.8.5.3. 重建分区
12.8.5.4. 分区维护
12.8.6. EXPLAIN PARTITIONS
12.8.7. SHOW CREATE TABLE
12.8.8. INFORMATION_SCHEMA.partitions 表
12.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
12.9. Engine
12.9.1. FEDERATED
12.9.2. BLACKHOLE
13. DML (Data Manipulation Language)
13.1. INSERT
13.1.1. INSERT INTO ... SELECT
14. SQL Statement Syntax
14.1. DISTINCT
14.2. group by
14.3. replace
14.4. OUTFILE/LOAD DATA INFILE
14.4.1. Export data to CSV from MySQL
14.5. SQL 92
14.6. Functions and Operators
14.6.1. COUNT
14.6.2. UUID()
14.6.3. String
14.6.3.1. LEFT/RIGHT
14.6.3.2. RPAD/LPAD
14.6.3.3. CONCAT
14.6.3.4. GROUP_CONCAT
14.6.3.5. replace
14.6.4. Date and Time
14.6.4.1. year/month/day hour:minite:second
14.6.4.2. Unix time
14.6.4.3. DATE_FORMAT
14.6.4.4. DATE_SUB/DATE_ADD
14.6.4.5. datediff / timediff
14.7. 事务处理和锁定语句
14.7.1. 排他锁
14.7.2. 表的加锁与解锁
14.7.3. 事务所用到的表
14.8. MySQL 专有命令
14.8.1. SQL_NO_CACHE
15. Optimization
15.1. Buffering and Caching
15.1.1. Query Cache SELECT Options
15.2. where 优化
16. MySQL Connectors
16.1. JDBC
16.2. ODBC
16.3. MySQL native driver for PHP - mysqlnd
16.4. python-mysqldb
17. MySQL GUI/Web Manager
17.1. HeidiSQL
17.2. Toad for MySQL Freeware
17.3. phpMyAdmin - MySQL web administration tool
17.4. Maatkit Essential command-line utilities for MySQL
18. Miscellaneous
18.1. Multi-Master Replication Manager for MySQL
18.2. HandlerSocket
18.3. MariaDB
18.4. Maatkit
19. FAQ
19.1. 数据库内容替换
19.2. 查看错误代码
19.2.1. ERROR 1153 (08S01) at line 3168: Got a packet bigger than 'max_allowed_packet' bytes
19.2.2. ERROR 1129 (00000): Host 'XXXXXX' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
19.3. 临时表是否需要建索引
19.4. Kill 脚本


第 8 章 MySQL Server

目录

8.1. MySQL Installation
8.1.1. Installation by apt-get under debian/ubuntu
8.1.2. Installation by source code
8.1.3. MySQL binary distribution
8.1.4. mysql-5.5.21-debian6.0-i686.deb
8.1.5. CentOS 6.2 + MySQL 5.5.25 (RPM)
8.1.6. mysql-admin
8.2. my.cnf
8.2.1. bind-address
8.2.2. Configuring Database Character Encoding
8.2.3. max_connections
8.2.4. storage-engine
8.2.5. max_allowed_packet
8.2.6. skip-name-resolve
8.2.7. 禁用TCP/IP链接
8.2.8. timeout
8.2.9. 与复制有关的参数
8.2.9.1. 定义 log-bin 文件名
8.2.9.2. expire-logs-days
8.2.10. 与 InnoDB 有关的配置项
8.2.11. Example for my.cnf
8.3. Replication
8.3.1. Master Slave
8.3.1.1. Master
8.3.1.2. Slave
8.3.1.3. Testing
8.3.1.4. 将现有数据库迁移到主从结构数据库
8.3.1.5. 主从复制安全问题
8.3.1.6. 主从不同步问题
8.3.2. Master Master(主主)
8.3.2.1. Master A
8.3.2.2. Master B
8.3.2.3. 将Master A 数据库 同步到 Master B 两端数据库内容保持一致
8.3.2.4. Master A - B 同步两端数据库
8.3.2.5. Master A 数据库解除只读权限
8.3.2.6. 查看主主的工作状态
8.3.3. 与复制有关的问题
8.3.3.1. mysql-bin 清理问题
8.3.3.2. Semisynchronous Replication
8.4. MySQL Custer
8.4.1. Management node (MGM node)
8.4.2. Data node
8.4.3. SQL node
8.4.4. Starting
8.4.5. Shutdown
8.4.6. Testing
8.5. MySQL Proxy
8.5.1. Ubuntu
8.5.2. CentOS
8.5.2.1. FAQ
8.6. Security
8.6.1. Firewall
8.6.2. 保护表字段
8.7. Monitoring
8.7.1. Analysis and Optimization
8.7.1.1. mytop - top like query monitor for MySQL
8.7.1.2. mtop - MySQL terminal based query monitor
8.7.1.3. innotop
8.7.1.4. mysqlreport
8.7.2. Munin
8.7.3. Cacti
8.7.4. Monitoring MySQL with SNMP

8.1. MySQL Installation

http://downloads.mysql.com/archives.php

8.1.1. Installation by apt-get under debian/ubuntu

安装环境 ubuntu 8.10

sudo apt-get install mysql-server

New password for the MySQL "root" user

         ┌──────────────────────┤ Configuring mysql-server-5.0 ├─────────────────────┐
         │ While not mandatory, it is highly recommended that you set a password for the MySQL administrative "root" user.  │
         │                                                                                                                  │
         │ If that field is left blank, the password will not be changed.                                                   │
         │                                                                                                                  │
         │ New password for the MySQL "root" user:                                                                          │
         │                                                                                                                  │
         │ ****____________________________________________________________________________________________________________ │
         │                                                                                                                  │
         │                                                      <Ok>                                                        │
         │                                                                                                                  │
         └─────────────────────────────────────────────────────────────┘

Repeat password for the MySQL "root" user

         ┌───┤ Configuring mysql-server-5.0 ├────┐
         │                                             │
         │                                             │
         │ Repeat password for the MySQL "root" user:  │
         │                                             │
         │ ****_______________________________________ │
         │                                             │
         │                   <Ok>                      │
         │                                             │
         └─────────────────────────┘

create database

create database example;

mysql> SHOW GRANTS;
+----------------------------------------------------------------------------------------------------------------------------------------+
| Grants for root@localhost                                                                                                              |
+----------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY PASSWORD '*C6325DAF39AE6CC34E960D3C65F1398FE467E1D0' WITH GRANT OPTION |
+----------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

GRANT ALL PRIVILEGES ON example.* TO 'dbuser'@'localhost' IDENTIFIED BY '******' WITH GRANT OPTION;
FLUSH PRIVILEGES;

./mysql -udbuser -p
Enter password:

./mysql -udbuser -p example < /tmp/example_china_copy.sql

./mysql -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.0.45 Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use example;
Database changed
mysql> show tables;

配置文件样本

debian:~# ls /usr/share/doc/mysql-server-5.0/examples/
my-huge.cnf.gz  my-innodb-heavy-4G.cnf.gz  my-large.cnf.gz  my-medium.cnf.gz  my-small.cnf  ndb_mgmd.cnf

8.1.2. Installation by source code

./configure \
--prefix=/usr/local/$MYSQL_DIR \
--enable-assembler \
--enable-local-infile \
--with-charset=utf8 \
--with-collation=utf8_general_ci \
--with-extra-charsets=none \
--with-openssl \
--with-pthread \
--with-unix-socket-path=/var/lib/mysql/mysql.sock \
--with-mysqld-user=mysql \
--with-mysqld-ldflags \
--with-client-ldflags \
--with-comment \
--with-big-tables \
--without-ndb-debug \
--without-docs \
--without-debug \
--without-bench

make && make install

/usr/local/$MYSQL_DIR/bin/mysql_install_db

other option

--without-isam
--without-innodb
--without-ndbcluster
--without-blackhole
--without-ibmdb2i
--without-federated
--without-example
--without-comment
--localstatedir=/usr/local/mysql/data

8.1.3. MySQL binary distribution

shell> groupadd mysql
shell> useradd -r -g mysql mysql
shell> cd /usr/local
shell> tar zxvf /path/to/mysql-VERSION-OS.tar.gz
shell> ln -s full-path-to-mysql-VERSION-OS mysql
shell> cd mysql
shell> chown -R mysql .
shell> chgrp -R mysql .
shell> scripts/mysql_install_db --user=mysql
shell> chown -R root .
shell> chown -R mysql data
# Next command is optional
shell> cp support-files/my-medium.cnf /etc/my.cnf
shell> bin/mysqld_safe --user=mysql &
# Next command is optional
shell> cp support-files/mysql.server /etc/init.d/mysql.server

install core database

[root@test mysql]# ./scripts/mysql_install_db
Installing MySQL system tables...
100428 23:16:20 [Warning] '--skip-locking' is deprecated and will be removed in a future release. Please use '--skip-external-locking' instead.
OK
Filling help tables...
100428 23:16:20 [Warning] '--skip-locking' is deprecated and will be removed in a future release. Please use '--skip-external-locking' instead.
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

./bin/mysqladmin -u root password 'new-password'
./bin/mysqladmin -u root -h db.example.com password 'new-password'

Alternatively you can run:
./bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd . ; ./bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd ./mysql-test ; perl mysql-test-run.pl

Please report any problems with the ./bin/mysqlbug script!

set root's password

[root@test mysql]# cp support-files/mysql.server /etc/init.d/mysqld
[root@test mysql]# /etc/init.d/mysqld start
Starting MySQL.                                            [  OK  ]

[root@test mysql]# ./bin/mysqladmin -u root password 'chen'
[root@test mysql]# ./bin/mysqladmin -u root -h db.example.com password 'chen'

test

[root@test mysql]# ./bin/mysql -uroot -pchen
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.1.45 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

8.1.4. mysql-5.5.21-debian6.0-i686.deb

sudo apt-get install libaio1

sudo groupadd mysql
sudo useradd -r -g mysql mysql

sudo dpkg -i mysql-5.5.21-debian6.0-i686.deb

cd /opt/mysql/
sudo chown -R mysql .
sudo chgrp -R mysql .

cd server-5.5/

sudo support-files/binary-configure

sudo chown -R mysql data

# Next command is optional
shell> cp support-files/my-medium.cnf /etc/my.cnf

shell> bin/mysqld_safe --user=mysql &

# Next command is optional
sudo cp support-files/mysql.server /etc/init.d/mysql

8.1.5. CentOS 6.2 + MySQL 5.5.25 (RPM)

准备下面的软件包

# ls -1
MySQL-client-5.5.25-1.el6.x86_64.rpm
MySQL-devel-5.5.25-1.el6.x86_64.rpm
MySQL-server-5.5.25-1.el6.x86_64.rpm
MySQL-shared-5.5.25-1.el6.x86_64.rpm
MySQL-shared-compat-5.5.25-1.el6.x86_64.rpm

使用 yum 本地安装 rpm, yum 可以帮你解决依赖于冲突

# yum localinstall MySQL-*

# /etc/init.d/mysql start
Starting MySQL... SUCCESS!

# /usr/bin/mysqladmin -u root password 'tUG26WSslP30bkbwtMhn'

8.1.6. mysql-admin

$ sudo apt-get install mysql-admin

运行mysql-admin

/usr/bin/mysql-admin

运行 mysql-query-browser

mysql-query-browser --query="SELECT * FROM users"


8.2. my.cnf

8.2.1. bind-address

bind-address = 0.0.0.0

8.2.2. Configuring Database Character Encoding

mysql> SHOW VARIABLES LIKE 'character_set_%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | latin1                     |
| character_set_connection | latin1                     |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | latin1                     |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

Server Character Set and Collation

shell> mysqld --character-set-server=latin1
shell> mysqld --character-set-server=latin1 \
           --collation-server=latin1_swedish_ci

$ vim /etc/mysql/my.cnf

[mysqld]
character-set-server=utf8
collation_server=utf8_general_ci
init_connect='SET NAMES utf8'

[client]
character-set-server=utf8

mysql --default-character-set=utf8 -u root -p

mysql> show variables like 'character%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

8.2.3. max_connections

[mysqld]
max_connections=250

8.2.4. storage-engine

[mysqld]
default-storage-engine=INNODB

8.2.5. max_allowed_packet

max_allowed_packet=500M

8.2.6. skip-name-resolve

跳过域名解析

# vim /etc/mysql/my.cnf

[mysqld]
skip-external-locking
skip-name-resolve

MySQL 登录缓慢，大量用户排队等待

mysql> SHOW FULL PROCESSLIST;
+-----+----------------------+------------------------+------+---------+------+-------+-----------------------+
| Id  | User                 | Host                   | db   | Command | Time | State | Info                  |
+-----+----------------------+------------------------+------+---------+------+-------+-----------------------+
| 718 | unauthenticated user | 192.168.3.124:42075    | NULL | Connect | NULL | login | NULL                  |
| 719 | unauthenticated user | 192.168.3.124:42073    | NULL | Connect | NULL | login | NULL                  |
| 720 | unauthenticated user | 192.168.3.124:42074    | NULL | Connect | NULL | login | NULL                  |
| 721 | unauthenticated user | 192.168.3.124:42077    | NULL | Connect | NULL | login | NULL                  |
| 722 | unauthenticated user | 192.168.3.124:42076    | NULL | Connect | NULL | login | NULL                  |
| 723 | unauthenticated user | 192.168.3.124:42079    | NULL | Connect | NULL | login | NULL                  |
| 724 | unauthenticated user | 192.168.3.124:42078    | NULL | Connect | NULL | login | NULL                  |
| 725 | unauthenticated user | 192.168.3.124:42081    | NULL | Connect | NULL | login | NULL                  |
| 726 | unauthenticated user | 192.168.3.124:42080    | NULL | Connect | NULL | login | NULL                  |
| 727 | unauthenticated user | 192.168.3.124:42082    | NULL | Connect | NULL | login | NULL                  |
| 728 | unauthenticated user | 192.168.3.124:42083    | NULL | Connect | NULL | login | NULL                  |
| 729 | unauthenticated user | 192.168.3.124:42085    | NULL | Connect | NULL | login | NULL                  |
| 730 | unauthenticated user | 192.168.3.124:42084    | NULL | Connect | NULL | login | NULL                  |
| 731 | unauthenticated user | 192.168.3.124:42086    | NULL | Connect | NULL | login | NULL                  |
| 732 | unauthenticated user | 192.168.3.124:42087    | NULL | Connect | NULL | login | NULL                  |
| 733 | unauthenticated user | 192.168.3.124:42088    | NULL | Connect | NULL | login | NULL                  |
| 734 | unauthenticated user | 192.168.3.124:42089    | NULL | Connect | NULL | login | NULL                  |
| 735 | unauthenticated user | 192.168.3.124:42090    | NULL | Connect | NULL | login | NULL                  |
| 736 | unauthenticated user | 192.168.3.124:42091    | NULL | Connect | NULL | login | NULL                  |
| 737 | unauthenticated user | 192.168.3.124:42092    | NULL | Connect | NULL | login | NULL                  |
| 738 | unauthenticated user | 192.168.3.124:42093    | NULL | Connect | NULL | login | NULL                  |
| 739 | unauthenticated user | 192.168.3.124:42094    | NULL | Connect | NULL | login | NULL                  |
| 740 | unauthenticated user | 192.168.3.124:42095    | NULL | Connect | NULL | login | NULL                  |
| 741 | unauthenticated user | 192.168.3.124:42096    | NULL | Connect | NULL | login | NULL                  |
| 742 | unauthenticated user | 192.168.3.124:42097    | NULL | Connect | NULL | login | NULL                  |
| 743 | unauthenticated user | 192.168.3.124:42098    | NULL | Connect | NULL | login | NULL                  |
| 744 | unauthenticated user | 192.168.3.124:42099    | NULL | Connect | NULL | login | NULL                  |
| 745 | unauthenticated user | 192.168.3.124:42100    | NULL | Connect | NULL | login | NULL                  |
| 746 | unauthenticated user | 192.168.3.124:42101    | NULL | Connect | NULL | login | NULL                  |
| 747 | unauthenticated user | 192.168.3.124:42102    | NULL | Connect | NULL | login | NULL                  |
| 748 | unauthenticated user | 192.168.3.124:42103    | NULL | Connect | NULL | login | NULL                  |
| 749 | unauthenticated user | 192.168.3.124:42104    | NULL | Connect | NULL | login | NULL                  |
| 750 | unauthenticated user | 192.168.3.124:42068    | NULL | Connect | NULL | login | NULL                  |
| 751 | unauthenticated user | 192.168.3.124:42064    | NULL | Connect | NULL | login | NULL                  |
| 752 | unauthenticated user | 192.168.3.124:42071    | NULL | Connect | NULL | login | NULL                  |
| 753 | unauthenticated user | 192.168.3.124:42072    | NULL | Connect | NULL | login | NULL                  |
| 754 | unauthenticated user | 192.168.3.124:42067    | NULL | Connect | NULL | login | NULL                  |
| 755 | unauthenticated user | 192.168.3.124:42070    | NULL | Connect | NULL | login | NULL                  |
| 756 | unauthenticated user | 192.168.3.124:42069    | NULL | Connect | NULL | login | NULL                  |
| 757 | unauthenticated user | 192.168.3.124:42065    | NULL | Connect | NULL | login | NULL                  |
| 758 | unauthenticated user | 192.168.3.124:42112    | NULL | Connect | NULL | login | NULL                  |
| 759 | unauthenticated user | 192.168.3.50:4872      | NULL | Connect | NULL | login | NULL                  |
| 761 | unauthenticated user | 192.168.3.40:36363     | NULL | Connect | NULL | login | NULL                  |
| 762 | neo                  | www.example.com:56200  | NULL | Query   |    0 | NULL  | SHOW FULL PROCESSLIST |
+-----+----------------------+------------------------+------+---------+------+-------+-----------------------+
44 rows in set (0.00 sec)

mysql> SHOW FULL PROCESSLIST;
+-----+----------------------+------------------------+------+---------+------+-------+-----------------------+
| Id  | User                 | Host                   | db   | Command | Time | State | Info                  |
+-----+----------------------+------------------------+------+---------+------+-------+-----------------------+
| 718 | unauthenticated user | 192.168.3.124:42075    | NULL | Connect | NULL | login | NULL                  |
| 719 | unauthenticated user | 192.168.3.124:42073    | NULL | Connect | NULL | login | NULL                  |
| 720 | unauthenticated user | 192.168.3.124:42074    | NULL | Connect | NULL | login | NULL                  |
| 721 | unauthenticated user | 192.168.3.124:42077    | NULL | Connect | NULL | login | NULL                  |
| 722 | unauthenticated user | 192.168.3.124:42076    | NULL | Connect | NULL | login | NULL                  |
| 723 | unauthenticated user | 192.168.3.124:42079    | NULL | Connect | NULL | login | NULL                  |
| 724 | unauthenticated user | 192.168.3.124:42078    | NULL | Connect | NULL | login | NULL                  |
| 725 | unauthenticated user | 192.168.3.124:42081    | NULL | Connect | NULL | login | NULL                  |
| 726 | unauthenticated user | 192.168.3.124:42080    | NULL | Connect | NULL | login | NULL                  |
| 727 | unauthenticated user | 192.168.3.124:42082    | NULL | Connect | NULL | login | NULL                  |
| 728 | unauthenticated user | 192.168.3.124:42083    | NULL | Connect | NULL | login | NULL                  |
| 729 | unauthenticated user | 192.168.3.124:42085    | NULL | Connect | NULL | login | NULL                  |
| 730 | unauthenticated user | 192.168.3.124:42084    | NULL | Connect | NULL | login | NULL                  |
| 731 | unauthenticated user | 192.168.3.124:42086    | NULL | Connect | NULL | login | NULL                  |
| 732 | unauthenticated user | 192.168.3.124:42087    | NULL | Connect | NULL | login | NULL                  |
| 733 | unauthenticated user | 192.168.3.124:42088    | NULL | Connect | NULL | login | NULL                  |
| 734 | unauthenticated user | 192.168.3.124:42089    | NULL | Connect | NULL | login | NULL                  |
| 735 | unauthenticated user | 192.168.3.124:42090    | NULL | Connect | NULL | login | NULL                  |
| 736 | unauthenticated user | 192.168.3.124:42091    | NULL | Connect | NULL | login | NULL                  |
| 737 | unauthenticated user | 192.168.3.124:42092    | NULL | Connect | NULL | login | NULL                  |
| 738 | unauthenticated user | 192.168.3.124:42093    | NULL | Connect | NULL | login | NULL                  |
| 739 | unauthenticated user | 192.168.3.124:42094    | NULL | Connect | NULL | login | NULL                  |
| 740 | unauthenticated user | 192.168.3.124:42095    | NULL | Connect | NULL | login | NULL                  |
| 741 | unauthenticated user | 192.168.3.124:42096    | NULL | Connect | NULL | login | NULL                  |
| 742 | unauthenticated user | 192.168.3.124:42097    | NULL | Connect | NULL | login | NULL                  |
| 743 | unauthenticated user | 192.168.3.124:42098    | NULL | Connect | NULL | login | NULL                  |
| 744 | unauthenticated user | 192.168.3.124:42099    | NULL | Connect | NULL | login | NULL                  |
| 745 | unauthenticated user | 192.168.3.124:42100    | NULL | Connect | NULL | login | NULL                  |
| 746 | unauthenticated user | 192.168.3.124:42101    | NULL | Connect | NULL | login | NULL                  |
| 747 | unauthenticated user | 192.168.3.124:42102    | NULL | Connect | NULL | login | NULL                  |
| 748 | unauthenticated user | 192.168.3.124:42103    | NULL | Connect | NULL | login | NULL                  |
| 749 | unauthenticated user | 192.168.3.124:42104    | NULL | Connect | NULL | login | NULL                  |
| 750 | unauthenticated user | 192.168.3.124:42068    | NULL | Connect | NULL | login | NULL                  |
| 751 | unauthenticated user | 192.168.3.124:42064    | NULL | Connect | NULL | login | NULL                  |
| 752 | unauthenticated user | 192.168.3.124:42071    | NULL | Connect | NULL | login | NULL                  |
| 753 | unauthenticated user | 192.168.3.124:42072    | NULL | Connect | NULL | login | NULL                  |
| 754 | unauthenticated user | 192.168.3.124:42067    | NULL | Connect | NULL | login | NULL                  |
| 755 | unauthenticated user | 192.168.3.124:42070    | NULL | Connect | NULL | login | NULL                  |
| 756 | unauthenticated user | 192.168.3.124:42069    | NULL | Connect | NULL | login | NULL                  |
| 757 | unauthenticated user | 192.168.3.124:42065    | NULL | Connect | NULL | login | NULL                  |
| 758 | unauthenticated user | 192.168.3.124:42112    | NULL | Connect | NULL | login | NULL                  |
| 759 | unauthenticated user | 192.168.3.50:4872      | NULL | Connect | NULL | login | NULL                  |
| 761 | unauthenticated user | 192.168.3.40:36363     | NULL | Connect | NULL | login | NULL                  |
| 762 | neo                  | www.example.com:56200  | NULL | Query   |    0 | NULL  | SHOW FULL PROCESSLIST |
+-----+----------------------+------------------------+------+---------+------+-------+-----------------------+
44 rows in set (0.00 sec)

解决方案 my.cnf 配置文件中加入skip-name-resolve

8.2.7. 禁用TCP/IP链接

[mysqld]
skip-networking

8.2.8. timeout

[mysqld]
wait_timeout = 30
interactive_timeout =30

如果你没有修改过MySQL的配置，缺省情况下，wait_timeout的初始值是28800。

wait_timeout过大有弊端，其体现就是MySQL里大量的SLEEP进程无法及时释放，拖累系统性能，不过也不能把这个指设置的过小，否则你可能会遭遇到“MySQL has gone away”之类的问题，通常来说，我觉得把wait_timeout设置为10是个不错的选择，但某些情况下可能也会出问题，比如说有一个CRON脚本，其中两次SQL查询的间隔时间大于10秒的话，那么这个设置就有问题了：

8.2.9. 与复制有关的参数

8.2.9.1. 定义 log-bin 文件名

log-bin=mysql-bin

8.2.9.2. expire-logs-days

bin log 保留时间

expire-logs-days = 30

8.2.10. 与 InnoDB 有关的配置项

innodb_file_per_table

配置后重启mysql运行下面命令将ibdata1拆分到tbl_name.ibd

OPTIMIZE TABLE tbl_name;

ls /var/lib/mysql/中查看 tbl_name.ibd文件

临时开启

SET @@global.innodb_file_per_table = 1;

8.2.11. Example for my.cnf

例 8.1. my.cnf

# Example MySQL config file for very large systems.
#
# This is for a large system with memory of 1G-2G where the system runs mainly
# MySQL.
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.

# The following options will be passed to all MySQL clients
[client]
#password	= your_password
port		= 3306
socket		= /var/lib/mysql/mysql.sock

# Here follows entries for some specific programs
character-set-server=utf8

# The MySQL server
[mysqld]
port		= 3306
socket		= /var/lib/mysql/mysql.sock
skip-external-locking
key_buffer_size = 384M
max_allowed_packet = 1M
table_open_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 32M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8

#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
log-bin=mysql-bin

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id	= 1

# Replication Slave (comment out master section to use this)
#
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id       = 2
#
# The replication master for this slave - required
#master-host     =   <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user     =   <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password =   <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port     =  <port>
#
# binary logging - not required for slaves, but recommended
#log-bin=mysql-bin
#
# binary logging format - mixed recommended
#binlog_format=mixed

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /var/lib/mysql
#innodb_data_file_path = ibdata1:2000M;ibdata2:10M:autoextend
#innodb_log_group_home_dir = /var/lib/mysql
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 384M
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 100M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

# Here follows entries for some specific programs
skip-name-resolve
default-storage-engine	= INNODB

character-set-server=utf8
collation_server=utf8_general_ci
init_connect='SET NAMES utf8'

max_connections			= 4096
max_connect_errors		= 10

pid-file				= mysql.pid
log 					= mysql.log
log-error 				= mysql_error.log

log-slow-queries 		= slow.log
long_query_time 		= 10

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout


8.3. Replication

8.3.1. Master Slave

8.3.1.1. Master

过程 8.1. Master 设置步骤

  1. 配置 my.cnf 文件

    确保主服务器主机上my.cnf文件的[mysqld]部分包括一个log-bin选项。该部分还应有一个server-id=Master_id选项

    # vim /etc/mysql/my.cnf

server-id               = 1
log_bin                 = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size         = 100M
binlog_do_db            = test
binlog_ignore_db        = mysql

    bind-address默认是127.0.0.1你必须更改它，否则Slave将无法链接到 Master

    #bind-address		= 127.0.0.1
bind-address		= 0.0.0.0

    重启服务器

    neo@netkiller:~$ sudo /etc/init.d/mysql reload
 * Reloading MySQL database server mysqld          [ OK ]

    建议使用reload,如果不起作用再用restart

  2. 登录slave服务器，测试主库3306工作情况，如果看到下面相关信息表示工作正常。

# telnet 192.168.1.246 3306
Trying 192.168.1.246...
Connected to 192.168.1.246.
Escape character is '^]'.
I
5.1.61-0ubuntu0.11.10.1-log1W<gs/*'#}p<u[J=5//:

  3. 创建账户并授予REPLICATION SLAVE权限

mysql> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%.mydomain.com' IDENTIFIED BY 'slavepass';
mysql> FLUSH PRIVILEGES;

    GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO replication@'192.168.245.131' IDENTIFIED BY 'slavepass'

  4. 锁表禁止写入新数据

mysql> FLUSH TABLES WITH READ LOCK;

  5. 查看Master 工作状态

mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000002 |      106 | test         | mysql            |
+------------------+----------+--------------+------------------+
1 row in set (0.00 sec)

    如果显示下面内容表示，配置不正确

mysql> SHOW MASTER STATUS;
Empty set (0.02 sec)

    取得快照并记录日志名和偏移量后，可以在主服务器上重新启用写活动

mysql> UNLOCK TABLES;

8.3.1.2. Slave

过程 8.2. Slave 设置步骤

  1. 配置my.cnf

    从服务器的ID必须与主服务器的ID不相同,如果设置多个从服务器，每个从服务器必须有一个唯一的server-id值，必须与主服务器的以及其它从服务器的不相同。

    # vim /etc/mysql/my.cnf

[mysqld]
server-id               = 2

  2. # service mysql restart
mysql start/running, process 22893

  3. 指定 master 相关参数

    在从服务器上执行下面的语句，用你的系统的实际值替换选项值

mysql> CHANGE MASTER TO
    ->     MASTER_HOST='master_host_name',
    ->     MASTER_USER='replication_user_name',
    ->     MASTER_PASSWORD='replication_password',
    ->     MASTER_LOG_FILE='recorded_log_file_name',
    ->     MASTER_LOG_POS=recorded_log_position;

    CHANGE MASTER TO MASTER_HOST='192.168.245.129', MASTER_USER='replication', MASTER_PASSWORD='slavepass';

mysql> CHANGE MASTER TO MASTER_HOST='192.168.245.129', MASTER_USER='repl', MASTER_PASSWORD='slavepass';
Query OK, 0 rows affected (0.14 sec)

  4. 启动从服务器线程

mysql> START SLAVE;
Query OK, 0 rows affected (0.00 sec)

  5. SLAVE STATUS

mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
             Slave_IO_State: Connecting to master
                Master_Host: 192.168.245.129
                Master_User: repl
                Master_Port: 3306
              Connect_Retry: 60
            Master_Log_File:
        Read_Master_Log_Pos: 4
             Relay_Log_File: mysqld-relay-bin.000002
              Relay_Log_Pos: 98
      Relay_Master_Log_File:
           Slave_IO_Running: Yes
          Slave_SQL_Running: Yes
            Replicate_Do_DB:
        Replicate_Ignore_DB:
         Replicate_Do_Table:
     Replicate_Ignore_Table:
    Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
                 Last_Errno: 0
                 Last_Error:
               Skip_Counter: 0
        Exec_Master_Log_Pos: 0
            Relay_Log_Space: 98
            Until_Condition: None
             Until_Log_File:
              Until_Log_Pos: 0
         Master_SSL_Allowed: No
         Master_SSL_CA_File:
         Master_SSL_CA_Path:
            Master_SSL_Cert:
          Master_SSL_Cipher:
             Master_SSL_Key:
      Seconds_Behind_Master: NULL
1 row in set (0.00 sec)

8.3.1.3. Testing

  1. 登录 master

    复制进程的信息

    SHOW PROCESSLIST语句可以提供在主服务器上和从服务器上发生的关于复制的信息

    mysql> SHOW PROCESSLIST\G
*************************** 1. row ***************************
     Id: 62
   User: replication
   Host: ken-hx409.local:36465
     db: NULL
Command: Binlog Dump
   Time: 679
  State: Has sent all binlog to slave; waiting for binlog to be updated
   Info: NULL
*************************** 2. row ***************************
     Id: 64
   User: root
   Host: localhost
     db: NULL
Command: Query
   Time: 0
  State: NULL
   Info: SHOW PROCESSLIST
2 rows in set (0.00 sec)

  2. 登录从库，查看复制线程

mysql> SHOW PROCESSLIST\G
*************************** 1. row ***************************
     Id: 273
   User: root
   Host: localhost
     db: NULL
Command: Query
   Time: 0
  State: NULL
   Info: SHOW PROCESSLIST
*************************** 2. row ***************************
     Id: 276
   User: system user
   Host:
     db: NULL
Command: Connect
   Time: 2
  State: Waiting for master to send event
   Info: NULL
*************************** 3. row ***************************
     Id: 277
   User: system user
   Host:
     db: NULL
Command: Connect
   Time: 2
  State: Has read all relay log; waiting for the slave I/O thread to update it
   Info: NULL
3 rows in set (0.00 sec)

    如果没有复制进程，请使用start slave;启动

mysql> SHOW PROCESSLIST\G
*************************** 1. row ***************************
     Id: 273
   User: root
   Host: localhost
     db: NULL
Command: Query
   Time: 0
  State: NULL
   Info: SHOW PROCESSLIST
1 row in set (0.02 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

  3. 登录 master

mysql> insert into foo(id,data) values(2,'Hello world!!!');
Query OK, 1 row affected (0.00 sec)

  4. 登录 slave

mysql> select * from foo;

    在master服务器上插入一条记录，你可以立刻在slave服务器上看到变化。

8.3.1.4. 将现有数据库迁移到主从结构数据库

数据库已经存在的情况下怎么迁移

  1. Master 锁表禁止写入新数据

mysql> FLUSH TABLES WITH READ LOCK;

  2. Slave 停止复制进程

mysql> stop slave;

  3. 备份Master数据库

mysqldump yourdb | gzip > yourdb.sql.gz

  4. 恢复数据库

    如果使用mysqldump备份主服务器的数据，将转储文件装载到从服务器

# zcat yourdb.sql.gz | mysql -u root -p yourdb

  5. 启动 Slave 复制进程

mysql> start slave;

  6. 解除 Master 表锁定

mysql> UNLOCK TABLES;

MyIASM引擎可以采用下面方法

备份数据库

# tar zcvf mysql-snapshot.tar.gz /var/lib/mysql/neo

复制给从数据库

scp mysql-snapshot.tar.gz  neo@192.168.245.131:/tmp

snapshot 恢复

$ tar zxvf mysql-snapshot.tar.gz
$ cd /var/lib/mysql

$ mv /tmp/var/lib/mysql/neo .
$ sudo chown mysql.mysql -R neo

重新启动Mysql

$ sudo /etc/init.d/mysql restart

有兴趣可以看看mysqlhotcopy

8.3.1.5. 主从复制安全问题

复制帐号权限

grant replication slave on *.* to 'replication'@'192.168.1.%' identified by '000000';

主库数据库操作帐号权限

grant DELETE, INSERT, SELECT, UPDATE ON your_user.* to yourdb@'your_host' identified by 'password' with grant option;

从库数据库操作帐号权限

grant SELECT ON your_user.* to yourdb@'your_host' identified by 'password' with grant option;

从库必须收回写操作

8.3.1.6. 主从不同步问题

执行下面语句

stop slave;
set global sql_slave_skip_counter =1 ;
start slave;

Seconds_Behind_Master 值从NULL变为大于等于0是表示已经同步

Seconds_Behind_Master: NULL
Seconds_Behind_Master: 8893

8.3.2. Master Master(主主)

8.3.2.1. Master A

my.cnf 文件加入下面的内容

cp /etc/mysql/my.cnf /etc/mysql/my.cnf.old
vim /etc/mysql/my.cnf

[mysqld]
server-id = 1
log-bin=/data/mysql/binlog/binlog
binlog-do-db = test
binlog-ignore-db=mysql

log-slave-updates
sync_binlog=1
auto_increment_offset=1
auto_increment_increment=2
replicate-do-db = test
replicate-ignore-db = mysql,information_schema

创建复制权限

grant replication slave on *.* to 'replication'@'192.168.1.%' identified by '000000';
flush privileges;

mysql>flush tables with read lock;

mysql> show master status\G
*************************** 1. row ***************************
File: binlog.000007
Position: 107
Binlog_Do_DB: test
Binlog_Ignore_DB: mysql
1 row in set (0.00 sec)

8.3.2.2. Master B

创建复制权限

grant replication slave on *.* to 'replication'@'192.168.1.%' identified by '000000';
flush privileges;

my.cnf 文件加入下面的内容

[mysqld]
server-id = 2
log-bin = /data/mysql/binlog/binlog
replicate-do-db = test
replicate-ignore-db = mysql,information_schema

binlog-do-db = test
binlog-ignore-db=mysql
log-slave-updates
sync_binlog=1
auto_increment_offset=2
auto_increment_increment=2

B 与 A 配置文件不同的两处。

server-id = 2
auto_increment_offset=2

mysql> show master status\G
*************************** 1. row ***************************
File: binlog.000005
Position: 107
Binlog_Do_DB: test
Binlog_Ignore_DB: mysql
1 row in set (0.00 sec)

8.3.2.3. 将Master A 数据库 同步到 Master B 两端数据库内容保持一致

Master A，首先锁表为只读状态

mysqldump --databases test > /tmp/test.sql

Master B 创建一个与Master A同名的空数据库,然后将备份文件恢复到数据库中

# mysql
mysql> CREATE DATABASE test;
mysql>\q

# scp 192.168.1.1:/tmp/test.sql ./
# mysql -uroot -p test < /tmp/test.sql

8.3.2.4. Master A - B 同步两端数据库

master-A

mysql>change master to master_host='192.168.1.2', master_user='replication', master_password='000000', master_log_file='binlog.000005', master_log_pos=107;

master-B

mysql>change master to master_host='192.168.1.1', master_user='replication', master_password='000000', master_log_file='binlog.000007', master_log_pos=107;

8.3.2.5. Master A 数据库解除只读权限

Master A 解锁

mysql> UNLOCK TABLES;

8.3.2.6. 查看主主的工作状态

分别在Master A与B 上运行

mysql>show slave status\G;

Slave_IO_Running: Yes
Slave_SQL_Running: Yes

8.3.3. 与复制有关的问题

8.3.3.1. mysql-bin 清理问题

缺省expire-logs-days为30天。这里设为7天，可根据自己情况调整。

[mysqld]
expire-logs-days = 7

通过SQL删除

删除某个日志：	mysql>PURGE MASTER LOGS TO ‘mysql-bin.015′;
删除某天前的日志：	mysql>PURGE MASTER LOGS BEFORE ’2010-10-25  14:00:00′;

8.3.3.2. Semisynchronous Replication

mysql> SHOW VARIABLES LIKE "have_dynamic_loading";
+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| have_dynamic_loading | YES   |
+----------------------+-------+
1 row in set (0.00 sec)

mysql>

master  > INSTALL PLUGIN rpl_semi_sync_master SONAME ‘libsemisync_master.so’;
slave-x > INSTALL PLUGIN rpl_semi_sync_slave SONAME ‘libsemisync_slave.so’;
master  > SET GLOBAL rpl_semi_sync_master_enabled=1;
slave-x > SET GLOBAL rpl_semi_sync_slave_enabled=1;

CREATE USER 'replication'@'192.168.1.%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON *.* TO 'replication'@'192.168.1.%' IDENTIFIED BY 'password' WITH GRANT OPTION MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ;

master_heartbeat_period

STOP SLAVE;
CHANGE MASTER TO master_heartbeat_period= milliseconds;
START SLAVE;

mysql> CHANGE MASTER TO MASTER_HOST='xxx.xxx.xxx.xxx', MASTER_USER='root', MASTER_PASSWORD='password', MASTER_PORT=3306, MASTER_CONNECT_RETRY=10, MASTER_LOG_FILE='mysql-bin.000004', MASTER_LOG_POS=107;

跳过出问题的MySQL实例

CHANGE MASTER TO MASTER_HOST=xxx.xxx.xxx.xxx IGNORE_SERVER_IDS=y

编辑my.cnf加入

# On Master
[mysqld]
rpl_semi_sync_master_enabled=1
rpl_semi_sync_master_timeout=1000 # 1 second

# On Slave
[mysqld]
rpl_semi_sync_slave_enabled=1


8.4. MySQL Custer

The cluster need a lot of server for experiments, if you haven't any server for one, I have a good idea that using Vmware for you.

at first, let's create lots of virtual machine(You MUST have a third server). and then follow me step by step learning how to set up a mysql cluster on your virtual machine.

mgm 		192.168.0.1		# Management
data 		192.168.0.2		# Ndbd Node
data 		192.168.0.3		# Ndbd Node
sql			192.168.0.4		# SQL Node
sql			192.168.0.5		# SQL Node

8.4.1. Management node (MGM node)

neo@mgm:~$ sudo vim /var/lib/mysql-cluster/config.ini

[NDBD DEFAULT]
NoOfReplicas=2
DataMemory=80M
IndexMemory=18M

[MYSQLD DEFAULT]

[NDB_MGMD DEFAULT]

[TCP DEFAULT]
portnumber=2202

[NDB_MGMD]
hostname=192.168.0.1
datadir=/var/lib/mysql-cluster

[NDBD]
hostname=192.168.0.2
datadir=/var/lib/mysql-cluster

[NDBD]
hostname=192.168.0.3
datadir=/var/lib/mysql-cluster

[MYSQLD]
hostname=192.168.0.4

[MYSQLD]
hostname=192.168.0.5

8.4.2. Data node

my.cnf

neo@data:~$ sudo vim /etc/mysql/my.cnf

[mysqld]
ndbcluster
ndb-connectstring=192.168.0.1	# the IP of the MANAGMENT SERVER
[mysql_cluster]
ndb-connectstring=192.168.0.1	# the IP of the MANAGMENT SERVER

8.4.3. SQL node

my.cnf

neo@sql:~$ sudo vim /etc/mysql/my.cnf

[mysqld]
ndbcluster
ndb-connectstring=192.168.0.1	# the IP of the MANAGMENT SERVER
[mysql_cluster]
ndb-connectstring=192.168.0.1	# the IP of the MANAGMENT SERVER

8.4.4. Starting

  1. starting mgm

    neo@mgm:~$ sudo ndb_mgmd -f /var/lib/mysql-cluster/config.ini

  2. initial ndbd

    neo@data:~$ sudo ndbd --initial

    首次运行需要 --initial 参数，以后不需要。

8.4.5. Shutdown

MGM

$ sudo ndb_mgm -e shutdown

8.4.6. Testing

neo@mgm:~$ ndb_mgm
-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2    @192.168.0.2  (Version: 5.0.51, Nodegroup: 0)
id=3    @192.168.0.3  (Version: 5.0.51, Nodegroup: 0, Master)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @192.168.0.1  (Version: 5.0.51)

[mysqld(API)]   2 node(s)
id=4    @192.168.0.4  (Version: 5.0.51)
id=5    @192.168.0.5  (Version: 5.0.51)

ndb_mgm>

与没有使用簇的MySQL相比，在MySQL簇内操作数据的方式没有太大的区别。

执行这类操作时应记住三点

  1. 表必须用ENGINE=NDB或ENGINE=NDBCLUSTER选项创建，或用ALTER TABLE选项更改，以使用NDB Cluster存储引擎在 Cluster内复制它们。如果使用mysqldump的输出从已有数据库导入表，可在文本编辑器中打开SQL脚本，并将该选项添加到任何表创建语句，或用这类选项之一替换任何已有的ENGINE（或TYPE）选项。

  2. 另外还请记住，每个NDB表必须有一个主键。如果在创建表时用户未定义主键，NDB Cluster存储引擎将自动生成隐含的主键。（注释：该隐含 键也将占用空间，就像任何其他的表索引一样。由于没有足够的内存来容纳这些自动创建的键，出现问题并不罕见）。

  3. 当你在一个节点上运行create database mydb;你去其他sql node上执行show databases;将不能看到mydb,你需要创建它，然后use mydb; show tables;你将看到同步的表。

SQL Node 1

neo@sql:~$ mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.0.51a-3ubuntu5.1 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create database cluster;
Query OK, 1 row affected (0.00 sec)

mysql> use cluster
Database changed
mysql> create table city( id mediumint unsigned not null auto_increment primary key, name varchar(20) not null default '' ) engine = ndbcluster default charset utf8;
Query OK, 0 rows affected (1.07 sec)

mysql> insert into city values(1, 'Shenzhen');
Query OK, 1 row affected (0.12 sec)

mysql> insert into city values(2, 'Guangdong');
Query OK, 1 row affected (0.00 sec)

SQL Node 2

neo@sql:~$ mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.0.51a-3ubuntu5.1 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| example            |
| mydb               |
| mysql              |
| neo                |
+--------------------+
6 rows in set (0.13 sec)

mysql> create database cluster;
Query OK, 1 row affected (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| cluster            |
| example            |
| mydb               |
| mysql              |
| neo                |
+--------------------+
6 rows in set (0.13 sec)

mysql> use cluster;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+-------------------+
| Tables_in_cluster |
+-------------------+
| city              |
+-------------------+
1 row in set (0.01 sec)

mysql> select * from city;
+----+-----------+
| id | name      |
+----+-----------+
|  1 | Shenzhen  |
|  2 | Guangdong |
+----+-----------+
2 rows in set (0.03 sec)

mysql>


8.5. MySQL Proxy

8.5.1. Ubuntu

安装环境 Ubuntu 13.04

$ sudo apt-get install mysql-proxy

ENABLED改为true

$ sudo vim /etc/default/mysql-proxy
ENABLED="true"
OPTIONS="--defaults-file=/etc/mysql/mysql-proxy.cnf"

配置 /etc/mysql/mysql-proxy.cnf

$ sudo vim /etc/mysql/mysql-proxy.cnf

[mysql-proxy]
daemon = true
user = mysql
proxy-skip-profiling = true
keepalive = true
max-open-files = 2048
event-threads = 50
pid-file = /var/run/mysql-proxy.pid
log-file = /var/log/mysql-proxy.log
log-level = debug
admin-address=:4401
admin-username=admin
admin-password=passw0rd
admin-lua-script=/usr/local/lib/mysql-proxy/lua/admin.lua
proxy-address = 0.0.0.0:3307
proxy-backend-addresses = 192.168.2.1:3306
proxy-read-only-backend-addresses=192.168.6.2:3306, 192.168.6.1:3306
proxy-lua-script=/usr/lib/mysql-proxy/lua/proxy/balance.lua

修改权限，这个步骤不能省略，否则无法启动。

$ sudo chmod 0660 /etc/mysql/mysql-proxy.cnf

启动mysql-proxy

$ sudo /etc/init.d/mysql-proxy start
 * Starting MySQL Proxy daemon...                    [ OK ]

测试3307端口

$ mysql -hlocalhost -P3307 -uroot -p

mysql-proxy 软件包所含文件如下：

$ dpkg -L  mysql-proxy
/.
/etc
/etc/default
/etc/default/mysql-proxy
/etc/init.d
/etc/init.d/mysql-proxy
/usr
/usr/share
/usr/share/mysql-proxy
/usr/share/mysql-proxy/active-queries.lua
/usr/share/mysql-proxy/active-transactions.lua
/usr/share/mysql-proxy/admin-sql.lua
/usr/share/mysql-proxy/admin.lua
/usr/share/mysql-proxy/analyze-query.lua
/usr/share/mysql-proxy/auditing.lua
/usr/share/mysql-proxy/commit-obfuscator.lua
/usr/share/mysql-proxy/histogram.lua
/usr/share/mysql-proxy/load-multi.lua
/usr/share/mysql-proxy/ro-balance.lua
/usr/share/mysql-proxy/ro-pooling.lua
/usr/share/mysql-proxy/rw-splitting.lua
/usr/share/mysql-proxy/xtab.lua
/usr/share/doc
/usr/share/doc/mysql-proxy
/usr/share/doc/mysql-proxy/README.TESTS.gz
/usr/share/doc/mysql-proxy/README
/usr/share/doc/mysql-proxy/copyright
/usr/share/doc/mysql-proxy/changelog.Debian.gz
/usr/lib
/usr/lib/libmysql-chassis-glibext.so.0.0.0
/usr/lib/libmysql-chassis-timing.so.0.0.0
/usr/lib/libmysql-chassis.so.0.0.0
/usr/lib/libmysql-proxy.so.0.0.0
/usr/lib/mysql-proxy
/usr/lib/mysql-proxy/lua
/usr/lib/mysql-proxy/lua/proxy
/usr/lib/mysql-proxy/lua/proxy/auto-config.lua
/usr/lib/mysql-proxy/lua/proxy/balance.lua
/usr/lib/mysql-proxy/lua/proxy/commands.lua
/usr/lib/mysql-proxy/lua/proxy/parser.lua
/usr/lib/mysql-proxy/lua/proxy/tokenizer.lua
/usr/lib/mysql-proxy/lua/proxy/test.lua
/usr/lib/mysql-proxy/lua/admin.lua
/usr/lib/mysql-proxy/lua/lfs.so
/usr/lib/mysql-proxy/lua/glib2.so
/usr/lib/mysql-proxy/lua/chassis.so
/usr/lib/mysql-proxy/lua/mysql.so
/usr/lib/mysql-proxy/lua/lpeg.so
/usr/lib/mysql-proxy/lua/posix.so
/usr/lib/mysql-proxy/plugins
/usr/lib/mysql-proxy/plugins/libadmin.so
/usr/lib/mysql-proxy/plugins/libproxy.so
/usr/lib/mysql-proxy/plugins/libreplicant.so
/usr/lib/mysql-proxy/plugins/libdebug.so
/usr/lib/pkgconfig
/usr/lib/pkgconfig/mysql-proxy.pc
/usr/lib/pkgconfig/mysql-chassis.pc
/usr/bin
/usr/bin/mysql-binlog-dump
/usr/bin/mysql-myisam-dump
/usr/bin/mysql-proxy
/usr/include
/usr/include/network-mysqld.h
/usr/include/network-mysqld-lua.h
/usr/include/network-mysqld-proto.h
/usr/include/network-mysqld-binlog.h
/usr/include/network-mysqld-packet.h
/usr/include/network-mysqld-masterinfo.h
/usr/include/network-conn-pool.h
/usr/include/network-conn-pool-lua.h
/usr/include/network-queue.h
/usr/include/network-socket.h
/usr/include/network-socket-lua.h
/usr/include/network-address.h
/usr/include/network-address-lua.h
/usr/include/sys-pedantic.h
/usr/include/chassis-plugin.h
/usr/include/chassis-log.h
/usr/include/chassis-keyfile.h
/usr/include/chassis-mainloop.h
/usr/include/chassis-path.h
/usr/include/chassis-filemode.h
/usr/include/chassis-limits.h
/usr/include/chassis-event-thread.h
/usr/include/chassis-gtimeval.h
/usr/include/glib-ext.h
/usr/include/glib-ext-ref.h
/usr/include/string-len.h
/usr/include/lua-load-factory.h
/usr/include/lua-scope.h
/usr/include/lua-env.h
/usr/include/network-injection.h
/usr/include/network-injection-lua.h
/usr/include/chassis-shutdown-hooks.h
/usr/include/chassis-exports.h
/usr/include/network-exports.h
/usr/include/network-backend.h
/usr/include/network-backend-lua.h
/usr/include/disable-dtrace.h
/usr/include/lua-registry-keys.h
/usr/include/chassis-stats.h
/usr/include/chassis-timings.h
/usr/include/chassis-frontend.h
/usr/include/chassis-options.h
/usr/include/chassis-win32-service.h
/usr/include/chassis-unix-daemon.h
/usr/include/my_rdtsc.h
/usr/lib/libmysql-chassis-glibext.so.0
/usr/lib/libmysql-chassis-glibext.so
/usr/lib/libmysql-proxy.so
/usr/lib/libmysql-chassis-timing.so.0
/usr/lib/libmysql-chassis-timing.so
/usr/lib/libmysql-proxy.so.0
/usr/lib/libmysql-chassis.so.0
/usr/lib/libmysql-chassis.so

8.5.2. CentOS

# yum install mysql-proxy

# cat /etc/sysconfig/mysql-proxy
# Options for mysql-proxy
ADMIN_USER="admin"
ADMIN_PASSWORD=""
ADMIN_LUA_SCRIPT="/usr/lib64/mysql-proxy/lua/admin.lua"
PROXY_USER="mysql-proxy"
PROXY_OPTIONS="--daemon --log-level=info --log-use-syslog"

修改PROXY_OPTIONS选项

#PROXY_OPTIONS="--daemon --log-level=info --log-use-syslog"
PROXY_OPTIONS="--defaults-file=/etc/mysql/mysql-proxy.cnf"

# mkdir /etc/mysql
# vim /etc/mysql/mysql-proxy.cnf

[mysql-proxy]
daemon = true
user = mysql-proxy
proxy-skip-profiling = true
keepalive = true
;max-open-files = 2048
event-threads = 512
pid-file = /var/run/mysql-proxy.pid
log-file = /var/log/mysql-proxy.log
log-level = debug
admin-address=:4401
admin-username=admin
admin-password=passw0rd
admin-lua-script=/usr/lib64/mysql-proxy/lua/admin.lua
proxy-address = 0.0.0.0:3307
proxy-backend-addresses = 192.168.2.1:3306
proxy-read-only-backend-addresses=192.168.6.2:3306, 192.168.6.1:3306
proxy-lua-script=/usr/lib64/mysql-proxy/lua/proxy/balance.lua

修复启动脚本BUG

# vim /etc/init.d/mysql-proxy

#daemon $prog $PROXY_OPTIONS --pid-file=$PROXY_PID --user=$PROXY_USER --admin-username="$ADMIN_USER" --admin-lua-script="$ADMIN_LUA_SCRIPT" --admin-password="$ADMIN_PASSWORD" 注视这行，改为下面的一行代码
daemon $prog $PROXY_OPTIONS --pid-file=$PROXY_PID

启动mysql-proxy

# chkconfig mysql-proxy on
# service mysql-proxy start
Starting mysql-proxy:                                      [  OK  ]

8.5.2.1. FAQ

(critical) (libevent) evsignal_init: socketpair: Too many open files

;max-open-files = 2048

注释max-open-files = 2048，使用ulimit -SHn 2048设置


8.6. Security

8.6.1. Firewall

iptables

iptables -A INPUT -i eth0 -p tcp -s xxx.xxx.xxx.xxx --dport 3306 -j ACCEPT

8.6.2. 保护表字段

通过触发器，使之无法修改某些字段的数据，同时不影响修改其他字段。

DROP TRIGGER IF EXISTS `members`;
SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='';
DELIMITER //
CREATE TRIGGER `members` BEFORE UPDATE ON `members` FOR EACH ROW BEGIN
    set new.name 		= old.name;
    set new.cellphone 	= old.cellphone;
    set new.email 		= old.email;
    set new.password 	= old.password;
END//
DELIMITER ;
SET SQL_MODE=@OLD_SQL_MODE;


8.7. Monitoring

http://netkiller.sourceforge.net/monitoring/index.html

8.7.1. Analysis and Optimization

8.7.1.1. mytop - top like query monitor for MySQL

sudo apt-get install mytop

mytop --host=172.16.0.7 --user=monitor --password=your_passwd

8.7.1.2. mtop - MySQL terminal based query monitor

http://mtop.sourceforge.net/

sudo apt-get install mtop
mtop --host=172.16.0.6 --dbuser=monitor --password=your_passwd

mkill

mkill -sl 180 -fi 'select.*from bad_table' > /var/log/mkill.out 2> /var/log/mkill.kill

8.7.1.3. innotop

8.7.1.4. mysqlreport

# yum install mysqlreport -y

wget hackmysql.com/scripts/mysqlreport

[root@database ~]# mysqlreport --user root --password chen
Use of uninitialized value in multiplication (*) at /usr/bin/mysqlreport line 829.
Use of uninitialized value in formline at /usr/bin/mysqlreport line 1227.
MySQL 5.0.77-log         uptime 28 23:42:33     Sat Apr 10 18:15:44 2010

__ Key _________________________________________________________________
Buffer used     6.54M of   8.00M  %Used:  81.75
  Current       1.49M            %Usage:  18.58
Write hit      97.65%
Read hit       99.81%

__ Questions ___________________________________________________________
Total           2.22M     0.9/s
  DMS           1.91M     0.8/s  %Total:  86.16
  Com_        249.93k     0.1/s           11.25
  COM_QUIT     63.68k     0.0/s            2.87
  -Unknown      6.26k     0.0/s            0.28
Slow 10 s          52     0.0/s            0.00  %DMS:   0.00  Log: OFF
DMS             1.91M     0.8/s           86.16
  SELECT        1.17M     0.5/s           52.81         61.29
  INSERT      276.13k     0.1/s           12.43         14.43
  DELETE      264.78k     0.1/s           11.92         13.84
  UPDATE      158.14k     0.1/s            7.12          8.26
  REPLACE      41.74k     0.0/s            1.88          2.18
Com_          249.93k     0.1/s           11.25
  set_option   89.09k     0.0/s            4.01
  change_db    59.71k     0.0/s            2.69
  show_create  28.57k     0.0/s            1.29

__ SELECT and Sort _____________________________________________________
Scan          161.33k     0.1/s %SELECT:  13.76
Range           6.47k     0.0/s            0.55
Full join       1.56k     0.0/s            0.13
Range check         0       0/s            0.00
Full rng join       0       0/s            0.00
Sort scan      34.03k     0.0/s
Sort range     21.98k     0.0/s
Sort mrg pass     733     0.0/s

__ Table Locks _________________________________________________________
Waited             56     0.0/s  %Total:   0.00
Immediate       2.15M     0.9/s

__ Tables ______________________________________________________________
Open               64 of   64    %Cache: 100.00
Opened        159.20k     0.1/s

__ Connections _________________________________________________________
Max used           36 of  200      %Max:  18.00
Total          63.75k     0.0/s

__ Created Temp ________________________________________________________
Disk table     32.80k     0.0/s
Table          63.69k     0.0/s    Size:  32.0M
File              319     0.0/s

__ Threads _____________________________________________________________
Running             1 of    1
Cached              0 of    0      %Hit:      0
Created        63.75k     0.0/s
Slow                0       0/s

__ Aborted _____________________________________________________________
Clients           128     0.0/s
Connects          130     0.0/s

__ Bytes _______________________________________________________________
Sent           23.89G    9.5k/s
Received        6.36G    2.5k/s

__ InnoDB Buffer Pool __________________________________________________
Usage           8.00M of   8.00M  %Used: 100.00
Read hit       99.99%
Pages
  Free              0            %Total:   0.00
  Data            511                     99.80 %Drty:   0.00
  Misc              1                      0.20
  Latched                                  0.00
Reads           1.54M     0.6/s
  From file       135     0.0/s            0.01
  Ahead Rnd         4     0.0/s
  Ahead Sql         6     0.0/s
Writes        868.00k     0.3/s
Flushes         1.56k     0.0/s
Wait Free           0       0/s

__ InnoDB Lock _________________________________________________________
Waits               0       0/s
Current             0
Time acquiring
  Total             0 ms
  Average           0 ms
  Max               0 ms

__ InnoDB Data, Pages, Rows ____________________________________________
Data
  Reads           194     0.0/s
  Writes          628     0.0/s
  fsync           323     0.0/s
  Pending
    Reads           0
    Writes          0
    fsync           0

Pages
  Created         534     0.0/s
  Read            201     0.0/s
  Written       1.56k     0.0/s

Rows
  Deleted           0       0/s
  Inserted    423.82k     0.2/s
  Read          1.27M     0.5/s
  Updated           0       0/s

8.7.2. Munin

8.7.3. Cacti

8.7.4. Monitoring MySQL with SNMP

mysql-snmp - monitoring MySQL with SNMP


第 9 章 Client and Utility Programs

目录

9.1. mysql - the MySQL command-line tool
9.1.1. ~/.my.cnf
9.1.2. 屏幕输出到文件
9.1.3. 终端编码
9.1.4. Unix Socket
9.1.5. 重定向巧用
9.2. mysqldump - a database backup program
9.2.1. 备份数据库，无结构，只有数据
9.2.2. 备份数据库结构（不备份数据）
9.2.3. 使用完整的insert插入数据
9.2.4. --extended-insert / --skip-extended-insert
9.2.5. --where
9.3. mysqladmin - client for administering a MySQL server
9.3.1. reload
9.3.2. 更改密码
9.3.3. status
9.3.4. process list
9.4. myisamchk — MyISAM Table-Maintenance Utility
9.5. mysqlcheck — A Table Maintenance and Repair Program
9.6. mysqlslap - load emulation client
9.7. mysqldumpslow - Parse and summarize the MySQL slow query log.
9.8. mysql log

9.1. mysql - the MySQL command-line tool

9.1.1. ~/.my.cnf

# mysql_secure_installation config file
[mysql]
user=root
password='chen'

9.1.2. 屏幕输出到文件

mysql>tee /home/neo/screen.txt
mysql>select * from member;
mysql>exit

9.1.3. 终端编码

mysql> show variables like 'char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

设置终端编码 set names utf8;

mysql> select * from category;
+----+------+-------------+--------+-----------+-----------+
| id | name | description | status | parent_id | path      |
+----+------+-------------+--------+-----------+-----------+
|  1 | ??   | ???????     | Y      |      NULL | 1/        |
|  4 | ???  | ???         | Y      |         1 | 1/4       |
|  5 | ???  | NULL        | Y      |         4 | 1/4/5     |
|  6 | ???  | NULL        | Y      |         5 | 1/4/5/6   |
|  7 | ???  | NULL        | Y      |         6 | 1/4/5/6/7 |
+----+------+-------------+--------+-----------+-----------+
5 rows in set (0.00 sec)

mysql> set names utf8;
Query OK, 0 rows affected (0.00 sec)

mysql> select * from category;
+----+-----------+-----------------------+--------+-----------+-----------+
| id | name      | description           | status | parent_id | path      |
+----+-----------+-----------------------+--------+-----------+-----------+
|  1 | 中国    | 中华人民共和家                                    | Y      |      NULL | 1/        |
|  4 | 广东省 | 广东省                                                      | Y      |         1 | 1/4       |
|  5 | 深圳市 | NULL                      | Y      |         4 | 1/4/5     |
|  6 | 宝安区 | NULL                      | Y      |         5 | 1/4/5/6   |
|  7 | 龙华镇 | NULL                      | Y      |         6 | 1/4/5/6/7 |
+----+-----------+-----------------------+--------+-----------+-----------+
5 rows in set (0.00 sec)

9.1.4. Unix Socket

mysql -uroot -p -S /tmp/mysql.sock

9.1.5. 重定向巧用

echo "show databases;" | mysql -uroot -pneo

cat |mysql -uroot -pneo << EOF
show databases;
EOF


9.2. mysqldump - a database backup program

mysqldump -uroot -p dbname | gzip > dbname.backup

9.2.1. 备份数据库，无结构，只有数据

-t, --no-create-info Don't write table creation info.

mysqldump -uroot -p -t -d database

9.2.2. 备份数据库结构（不备份数据）

mysqldump -uroot -p -d database
mysqldump -uroot -p -d database table

9.2.3. 使用完整的insert插入数据

-c, --complete-insert Use complete insert statements.

$ mysqldump -hlocalhost -uroot -t neo test

INSERT INTO `test` VALUES (98,'neo','chen'),(112,'jam','zheng'),(113,'john','meng');

$ mysqldump -hlocalhost -uroot -c -t neo test
INSERT INTO `test` (`userid`, `username`, `password`) VALUES (98,'neo','chen'),(112,'jam','zheng'),(113,'john','meng');

9.2.4. --extended-insert / --skip-extended-insert

--extended-insert 默认开启

INSERT INTO `test` VALUES (98,'neo','chen'),(112,'jam','zheng'),(113,'john','meng');

每条记录使用一次insert

$ mysqldump -hlocalhost -uroot --skip-extended-insert -t neo test |more
INSERT INTO `test` VALUES (98,'neo','chen');
INSERT INTO `test` VALUES (111,'neo','chen');
INSERT INTO `test` VALUES (112,'jam','zheng');
INSERT INTO `test` VALUES (113,'john','meng');

9.2.5. --where

mysqldump -hlocalhost -umysql -ppasswd database table --where="id>128"


9.3. mysqladmin - client for administering a MySQL server

9.3.1. reload

mysqladmin --user=root --password reload

9.3.2. 更改密码

sudo mysqladmin -u root -p password '你的新密码'

9.3.3. status

每个10秒输出一次mysql的状态信息

mysqladmin -i 10 extended status

mysqladmin -h 172.16.0.1 -u monitor -ppasswd status
Uptime: 195824  Threads: 21  Questions: 57744081  Slow queries: 516230  Opens: 13202607  Flush tables: 1  Open tables: 160  Queries per second avg: 294.877

 mysqladmin -h 172.16.0.1 -u monitor -ppasswd  extended-status
+-----------------------------------+-------------+
| Variable_name                     | Value       |
+-----------------------------------+-------------+
| Aborted_clients                   | 60336       |
| Aborted_connects                  | 4599        |
| Binlog_cache_disk_use             | 36          |
| Binlog_cache_use                  | 100721      |
| Bytes_received                    | 17510873261 |
| Bytes_sent                        | 92890743568 |
| Com_admin_commands                | 10026660    |
| Com_assign_to_keycache            | 0           |
| Com_alter_db                      | 0           |
| Com_alter_db_upgrade              | 0           |
| Com_alter_event                   | 0           |
| Com_alter_function                | 0           |
| Com_alter_procedure               | 0           |
| Com_alter_server                  | 0           |
| Com_alter_table                   | 418         |
| Com_alter_tablespace              | 0           |
| Com_analyze                       | 0           |
| Com_backup_table                  | 0           |
| Com_begin                         | 0           |
| Com_binlog                        | 0           |
| Com_call_procedure                | 0           |
| Com_change_db                     | 4440400     |
| Com_change_master                 | 1           |
| Com_check                         | 0           |
| Com_checksum                      | 0           |
| Com_commit                        | 30089       |
| Com_create_db                     | 1           |
| Com_create_event                  | 0           |
| Com_create_function               | 0           |
| Com_create_index                  | 1           |
| Com_create_procedure              | 0           |
| Com_create_server                 | 0           |
| Com_create_table                  | 211         |
| Com_create_trigger                | 4           |
| Com_create_udf                    | 0           |
| Com_create_user                   | 0           |
| Com_create_view                   | 2           |
| Com_dealloc_sql                   | 0           |
| Com_delete                        | 36664       |
| Com_delete_multi                  | 0           |
| Com_do                            | 0           |
| Com_drop_db                       | 0           |
| Com_drop_event                    | 0           |
| Com_drop_function                 | 0           |
| Com_drop_index                    | 0           |
| Com_drop_procedure                | 0           |
| Com_drop_server                   | 0           |
| Com_drop_table                    | 213         |
| Com_drop_trigger                  | 0           |
| Com_drop_user                     | 0           |
| Com_drop_view                     | 4           |
| Com_empty_query                   | 0           |
| Com_execute_sql                   | 0           |
| Com_flush                         | 9           |
| Com_grant                         | 6           |
| Com_ha_close                      | 0           |
| Com_ha_open                       | 0           |
| Com_ha_read                       | 0           |
| Com_help                          | 0           |
| Com_insert                        | 472260      |
| Com_insert_select                 | 0           |
| Com_install_plugin                | 0           |
| Com_kill                          | 12          |
| Com_load                          | 0           |
| Com_load_master_data              | 0           |
| Com_load_master_table             | 0           |
| Com_lock_tables                   | 209         |
| Com_optimize                      | 0           |
| Com_preload_keys                  | 0           |
| Com_prepare_sql                   | 0           |
| Com_purge                         | 0           |
| Com_purge_before_date             | 0           |
| Com_release_savepoint             | 0           |
| Com_rename_table                  | 0           |
| Com_rename_user                   | 0           |
| Com_repair                        | 0           |
| Com_replace                       | 4612        |
| Com_replace_select                | 0           |
| Com_reset                         | 0           |
| Com_restore_table                 | 0           |
| Com_revoke                        | 0           |
| Com_revoke_all                    | 0           |
| Com_rollback                      | 0           |
| Com_rollback_to_savepoint         | 0           |
| Com_savepoint                     | 0           |
| Com_select                        | 20310686    |
| Com_set_option                    | 9089818     |
| Com_show_authors                  | 0           |
| Com_show_binlog_events            | 0           |
| Com_show_binlogs                  | 0           |
| Com_show_charsets                 | 24          |
| Com_show_collations               | 18214       |
| Com_show_column_types             | 0           |
| Com_show_contributors             | 0           |
| Com_show_create_db                | 0           |
| Com_show_create_event             | 0           |
| Com_show_create_func              | 0           |
| Com_show_create_proc              | 0           |
| Com_show_create_table             | 0           |
| Com_show_create_trigger           | 0           |
| Com_show_databases                | 24          |
| Com_show_engine_logs              | 0           |
| Com_show_engine_mutex             | 0           |
| Com_show_engine_status            | 0           |
| Com_show_events                   | 0           |
| Com_show_errors                   | 0           |
| Com_show_fields                   | 147160      |
| Com_show_function_status          | 3           |
| Com_show_grants                   | 0           |
| Com_show_keys                     | 2           |
| Com_show_master_status            | 1           |
| Com_show_new_master               | 0           |
| Com_show_open_tables              | 0           |
| Com_show_plugins                  | 0           |
| Com_show_privileges               | 0           |
| Com_show_procedure_status         | 3           |
| Com_show_processlist              | 12483       |
| Com_show_profile                  | 0           |
| Com_show_profiles                 | 0           |
| Com_show_slave_hosts              | 0           |
| Com_show_slave_status             | 0           |
| Com_show_status                   | 1158        |
| Com_show_storage_engines          | 0           |
| Com_show_table_status             | 2           |
| Com_show_tables                   | 29915       |
| Com_show_triggers                 | 0           |
| Com_show_variables                | 26295       |
| Com_show_warnings                 | 0           |
| Com_slave_start                   | 0           |
| Com_slave_stop                    | 0           |
| Com_stmt_close                    | 0           |
| Com_stmt_execute                  | 0           |
| Com_stmt_fetch                    | 0           |
| Com_stmt_prepare                  | 0           |
| Com_stmt_reprepare                | 0           |
| Com_stmt_reset                    | 0           |
| Com_stmt_send_long_data           | 0           |
| Com_truncate                      | 0           |
| Com_uninstall_plugin              | 0           |
| Com_unlock_tables                 | 209         |
| Com_update                        | 501411      |
| Com_update_multi                  | 23112       |
| Com_xa_commit                     | 0           |
| Com_xa_end                        | 0           |
| Com_xa_prepare                    | 0           |
| Com_xa_recover                    | 0           |
| Com_xa_rollback                   | 0           |
| Com_xa_start                      | 0           |
| Compression                       | OFF         |
| Connections                       | 4555052     |
| Created_tmp_disk_tables           | 421231      |
| Created_tmp_files                 | 1172        |
| Created_tmp_tables                | 2769149     |
| Delayed_errors                    | 0           |
| Delayed_insert_threads            | 0           |
| Delayed_writes                    | 0           |
| Flush_commands                    | 1           |
| Handler_commit                    | 100721      |
| Handler_delete                    | 133583      |
| Handler_discover                  | 0           |
| Handler_prepare                   | 0           |
| Handler_read_first                | 404032      |
| Handler_read_key                  | 18292439681 |
| Handler_read_next                 | 33393351305 |
| Handler_read_prev                 | 77792315    |
| Handler_read_rnd                  | 2969739139  |
| Handler_read_rnd_next             | 41965058450 |
| Handler_rollback                  | 0           |
| Handler_savepoint                 | 0           |
| Handler_savepoint_rollback        | 0           |
| Handler_update                    | 4595750766  |
| Handler_write                     | 6069006380  |
| Innodb_buffer_pool_pages_data     | 19          |
| Innodb_buffer_pool_pages_dirty    | 0           |
| Innodb_buffer_pool_pages_flushed  | 0           |
| Innodb_buffer_pool_pages_free     | 493         |
| Innodb_buffer_pool_pages_misc     | 0           |
| Innodb_buffer_pool_pages_total    | 512         |
| Innodb_buffer_pool_read_ahead_rnd | 1           |
| Innodb_buffer_pool_read_ahead_seq | 0           |
| Innodb_buffer_pool_read_requests  | 77          |
| Innodb_buffer_pool_reads          | 12          |
| Innodb_buffer_pool_wait_free      | 0           |
| Innodb_buffer_pool_write_requests | 0           |
| Innodb_data_fsyncs                | 3           |
| Innodb_data_pending_fsyncs        | 0           |
| Innodb_data_pending_reads         | 0           |
| Innodb_data_pending_writes        | 0           |
| Innodb_data_read                  | 2494464     |
| Innodb_data_reads                 | 25          |
| Innodb_data_writes                | 3           |
| Innodb_data_written               | 1536        |
| Innodb_dblwr_pages_written        | 0           |
| Innodb_dblwr_writes               | 0           |
| Innodb_log_waits                  | 0           |
| Innodb_log_write_requests         | 0           |
| Innodb_log_writes                 | 1           |
| Innodb_os_log_fsyncs              | 3           |
| Innodb_os_log_pending_fsyncs      | 0           |
| Innodb_os_log_pending_writes      | 0           |
| Innodb_os_log_written             | 512         |
| Innodb_page_size                  | 16384       |
| Innodb_pages_created              | 0           |
| Innodb_pages_read                 | 19          |
| Innodb_pages_written              | 0           |
| Innodb_row_lock_current_waits     | 0           |
| Innodb_row_lock_time              | 0           |
| Innodb_row_lock_time_avg          | 0           |
| Innodb_row_lock_time_max          | 0           |
| Innodb_row_lock_waits             | 0           |
| Innodb_rows_deleted               | 0           |
| Innodb_rows_inserted              | 0           |
| Innodb_rows_read                  | 0           |
| Innodb_rows_updated               | 0           |
| Key_blocks_not_flushed            | 0           |
| Key_blocks_unused                 | 6917        |
| Key_blocks_used                   | 53585       |
| Key_read_requests                 | 35870213140 |
| Key_reads                         | 13788784    |
| Key_write_requests                | 35265303    |
| Key_writes                        | 2467239     |
| Last_query_cost                   | 0.000000    |
| Max_used_connections              | 3001        |
| Not_flushed_delayed_rows          | 0           |
| Open_files                        | 238         |
| Open_streams                      | 0           |
| Open_table_definitions            | 228         |
| Open_tables                       | 160         |
| Opened_files                      | 20864567    |
| Opened_table_definitions          | 653         |
| Opened_tables                     | 13202613    |
| Prepared_stmt_count               | 0           |
| Qcache_free_blocks                | 10480       |
| Qcache_free_memory                | 38697120    |
| Qcache_hits                       | 17943956    |
| Qcache_inserts                    | 8251298     |
| Qcache_lowmem_prunes              | 560647      |
| Qcache_not_cached                 | 11879434    |
| Qcache_queries_in_cache           | 54611       |
| Qcache_total_blocks               | 125193      |
| Queries                           | 57755205    |
| Questions                         | 57582352    |
| Rpl_status                        | NULL        |
| Select_full_join                  | 602236      |
| Select_full_range_join            | 6851        |
| Select_range                      | 1633467     |
| Select_range_check                | 0           |
| Select_scan                       | 10981650    |
| Slave_open_temp_tables            | 0           |
| Slave_retried_transactions        | 0           |
| Slave_running                     | OFF         |
| Slow_launch_threads               | 206         |
| Slow_queries                      | 516237      |
| Sort_merge_passes                 | 548         |
| Sort_range                        | 293328      |
| Sort_rows                         | 2831414035  |
| Sort_scan                         | 2726547     |
| Ssl_accept_renegotiates           | 0           |
| Ssl_accepts                       | 0           |
| Ssl_callback_cache_hits           | 0           |
| Ssl_cipher                        |             |
| Ssl_cipher_list                   |             |
| Ssl_client_connects               | 0           |
| Ssl_connect_renegotiates          | 0           |
| Ssl_ctx_verify_depth              | 0           |
| Ssl_ctx_verify_mode               | 0           |
| Ssl_default_timeout               | 0           |
| Ssl_finished_accepts              | 0           |
| Ssl_finished_connects             | 0           |
| Ssl_session_cache_hits            | 0           |
| Ssl_session_cache_misses          | 0           |
| Ssl_session_cache_mode            | NONE        |
| Ssl_session_cache_overflows       | 0           |
| Ssl_session_cache_size            | 0           |
| Ssl_session_cache_timeouts        | 0           |
| Ssl_sessions_reused               | 0           |
| Ssl_used_session_cache_entries    | 0           |
| Ssl_verify_depth                  | 0           |
| Ssl_verify_mode                   | 0           |
| Ssl_version                       |             |
| Table_locks_immediate             | 46406490    |
| Table_locks_waited                | 1428430     |
| Tc_log_max_pages_used             | 0           |
| Tc_log_page_size                  | 0           |
| Tc_log_page_waits                 | 0           |
| Threads_cached                    | 33          |
| Threads_connected                 | 33          |
| Threads_created                   | 77809       |
| Threads_running                   | 7           |
| Uptime                            | 195854      |
| Uptime_since_flush_status         | 195854      |
+-----------------------------------+-------------+

9.3.4. process list

[root@development ~]# mysqladmin -u root -p -h 127.0.0.1 processlist
Enter password:
+-------+--------+----------------------+--------+---------+------+-------+------------------+
| Id    | User   | Host                 | db     | Command | Time | State | Info             |
+-------+--------+----------------------+--------+---------+------+-------+------------------+
| 23648 | dbuser | 192.168.3.237:1220   | testdb | Sleep   | 2733 |       |                  |
| 23878 | dbuser | www.testdb.com:53639 | testdb | Sleep   | 7    |       |                  |
| 23881 | root   | localhost:57243      |        | Query   | 0    |       | show processlist |
+-------+--------+----------------------+--------+---------+------+-------+------------------+

mysql -u root -pneo -S /tmp/mysql.sock -e "show full processlist;"|grep -v Sleep


9.4. myisamchk — MyISAM Table-Maintenance Utility

先停止mysqld，在--datadir目录运行

myisamchk */*.MYI >/dev/null ＃检查哪些表需要修复

修复用以下命令一个个修复：

myisamchk -r table.MYI

更省事的做法：

myisamchk -r /var/lib/mysql/*.MYI

myisamchk可用crontab定時最佳化table

0 * * 0 /usr/bin/myisamchk -s /var/lib/mysql/*/*.MYI


9.5. mysqlcheck — A Table Maintenance and Repair Program

即可最佳化所有db

mysqlcheck -a -c -o -r --all-databases　-uroot -p
-a = Analyse given tables.
-c = Check table for errors
-o = Optimise table
-r = Can fix almost anything except unique keys that aren't unique

mysqlcheck -A -o -r -p


9.6. mysqlslap - load emulation client

–auto-generate-sql, -a
自动生成测试表和数据

–auto-generate-sql-load-type=type
测试语句的类型。取值包括：read，key，write，update和mixed(默认)。

–number-char-cols=N, -x N
自动生成的测试表中包含多少个字符类型的列，默认1

–number-int-cols=N, -y N
自动生成的测试表中包含多少个数字类型的列，默认1

–number-of-queries=N
总的测试查询次数(并发客户数×每客户查询次数)

–query=name,-q
使用自定义脚本执行测试，例如可以调用自定义的一个存储过程或者sql语句来执行测试。

–create-schema
测试的schema，MySQL中schema也就是database

–commint=N
多少条DML后提交一次

–compress, -C
如果服务器和客户端支持都压缩，则压缩信息传递

–concurrency=N, -c N
并发量，也就是模拟多少个客户端同时执行select。可指定多个值，以逗号或者–delimiter参数指定的值做为分隔符

–engine=engine_name, -e engine_name
创建测试表所使用的存储引擎，可指定多个

–iterations=N, -i N
测试执行的迭代次数

–detach=N
执行N条语句后断开重连

–debug-info, -T
打印内存和CPU的信息

–only-print
只打印测试语句而不实际执行

mysqlslap -u root -p -h localhost -c 10,50,100,200 -i 1 \
--engine=myisam --auto-generate-sql-load-type=mixed --number-of-queries=50000 \
--number-char-cols=5 --number-int-cols=5 --auto-generate-sql

mysqlslap --defaults-file=/etc/my.cnf --concurrency=50,100,200 --iterations=1 \
--number-int-cols=4 --number-char-cols=4 --auto-generate-sql --auto-generate-sql-add-autoincrement \
--auto-generate-sql-load-type=mixed --engine=myisam,innodb --number-of-queries=200 --debug-info \
-uroot -p -S/tmp/mysql.sock


9.7. mysqldumpslow - Parse and summarize the MySQL slow query log.

开启记录日志，修改my.cnf加入下面几行

--log-slow-queries[=file_name]

long_query_time = 10
log-slow-queries =

long_query_time 是指执行超过10秒的sql会被记录下来。

log-slow-queries设置把日志文件的位置，如果没有给出文件名值， 默认未主机名，后缀为-slow.log。如果给出了文件名，但不是绝对路径名，文件则写入数据目录。

cat /etc/mysql/my.cnf

[mysqld]
set-variable=long_query_time=1
log-slow-queries=/var/log/mysql/log-slow-queries.log

You must create the file manually and change owners this way:

touch /var/log/mysql/log-slow-queries.log
chown mysql.mysql -R /var/log/mysql/log-slow-queries.log

$ mysqldumpslow /var/log/mysql/log-slow-queries.log

mysqldumpslow 参数

  1. -s，是order的顺序，说明写的不够详细，俺用下来，包括看了代码，主要有c,t,l,r和ac,at,al,ar，t=time, l=lock time, r=rows分别是按照query次数，时间，lock的时间和返回的记录数来排序，前面加了a的时倒叙

  2. -t，是top n的意思，即为返回前面多少条的数据

  3. -g，后边可以写一个正则匹配模式，大小写不敏感的

  4. -g，后边可以写一个正则匹配模式，大小写不敏感的

mysqldumpslow -s c -t 20 ubuntu-slow.log

mysqldumpslow -s r -t 20 ubuntu-slow.log


9.8. mysql log

/etc/my.cnf 配置文件

在服务器上的/etc/my.cnf中的[client]加入
tee =/tmp/mysql_history.log即可.

查看log设置

show VARIABLES like '%log%';

命令行

mysql -uroot --tee=/tmp/mysql_history.log

mysql> tee mysql_history.log
Logging to file 'mysql_history.log '

或者
mysql> \T mysql_history.log
Logging to file 'mysql_history.log '

mysql> notee
Outfile disabled.
或者
mysql> \t
Outfile disabled.


第 10 章 Backup and Recovery

目录

10.1. Import / Export
10.1.1. Export(Backup)
10.1.2. Import(Recovery)
10.1.3. xml
10.1.4. 备份表数据
10.1.5. source
10.1.6. 使用 mysqlhotcopy 备份 MyISAM 引擎的数据库
10.1.7. AutoMySQLBackup
10.1.8. xtrabackup - Open source backup tool for InnoDB and XtraDB.
10.1.8.1. Percona yum Repository
10.1.8.2. Creating an Incremental Backup
10.2. Snapshot Backup
10.2.1. LVM Snapshot
10.2.2. Btrfs Snapshot

10.1. Import / Export

10.1.1. Export(Backup)

mysqldump -hlocalhost -proot -p**** mydb > mydb.sql

gzip

mysqldump -hlocalhost -proot -p**** mydb | gzip > mydb.sql.gz

10.1.2. Import(Recovery)

mysql -hlocalhost -proot -p**** mydb < mydb.sql

gunzip

gunzip mydb.sql.gz -c | mysql -hlocalhost -proot -p**** mydb

10.1.3. xml

export xml

$ mysqldump -uusrname -ppasswd -X -t database table -r filename.xml

10.1.4. 备份表数据

SELECT * INTO OUTFILE 'file_name' FROM tbl_name
LOAD DATA INFILE 'file_name' REPLACE INTO TABLE tbl_name

10.1.5. source

mysql> use your_db
mysql> SOURCE database.sql

10.1.6. 使用 mysqlhotcopy 备份 MyISAM 引擎的数据库

shell> mysqlhotcopy db_name /path/to/some/dir
mysql:~# mysqlhotcopy --user=neo --password=chen shop /tmp/backup
Locked 100 tables in 0 seconds.
Flushed tables (`shop`.`account_log`, `shop`.`ad`, `shop`.`ad_custom`, `shop`.`ad_position`, `shop`.`admin_action`,
`shop`.`admin_log`, `shop`.`admin_message`, `shop`.`admin_user`, `shop`.`adsense`, `shop`.`affiliate_log`,
...
...
...
`shop`.`user_rank`, `shop`.`users`, `shop`.`virtual_card`, `shop`.`volume_price`, `shop`.`vote`, `shop`.`vote_log`,
`shop`.`vote_option`, `shop`.`wholesale`) in 0 seconds.
Copying 299 files...
Copying indices for 0 files...
Unlocked tables.
mysqlhotcopy copied 100 tables (299 files) in 0 seconds (0 seconds overall).

10.1.7. AutoMySQLBackup

https://sourceforge.net/projects/automysqlbackup/

10.1.8. xtrabackup - Open source backup tool for InnoDB and XtraDB.

https://launchpad.net/percona-xtrabackup

10.1.8.1. Percona yum Repository

$ rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm

# yum search xtrabackup
================================================================ N/S Matched: XtraBackup =================================================================
holland-xtrabackup.noarch : Xtrabackup plugin for Holland
percona-xtrabackup.x86_64 : XtraBackup online backup for MySQL / InnoDB
percona-xtrabackup-debuginfo.x86_64 : Debug information for package percona-xtrabackup
percona-xtrabackup-test.x86_64 : Test suite for Percona Xtrabackup

# yum install percona-xtrabackup

10.1.8.2. Creating an Incremental Backup

xtrabackup --backup --target-dir=/data/backups/base --datadir=/var/lib/mysql/


10.2. Snapshot Backup

10.2.1. LVM Snapshot

# mysql –uroot –pmysql
mysql> flush tables with read lock;
mysql>flush logs;
mysql>system lvcreate -L1024M -s -n snap0 /dev/vg00/lvol00
mysql>show master status；
mysql>unlock tables；
mysql>quit

10.2.2. Btrfs Snapshot

# btrfs subvolume snapshot /data /data/backup_2013-03-20
Create a snapshot of '/data' in '/data/backup_2013-03-20'

btrfs subvolume list /data
ID 315 gen 172 top level 5 path backup_2013-03-10
ID 320 gen 178 top level 5 path backup_2013-03-20


第 11 章 Database Administration

目录

11.1. User Account Management
11.1.1. Create User
11.1.2. Drop User
11.1.3. Rename User
11.1.4. SET PASSWORD
11.1.5. Reset root password
11.2. Access Privilege System
11.2.1. SHOW GRANTS
11.2.2. show privileges
11.2.3. Grant privileges
11.2.4. Revoke privileges
11.2.5. Show Privileges
11.2.6. MAX_QUERIES_PER_HOUR/MAX_UPDATES_PER_HOUR
11.3. Maintenance 数据库维护
11.3.1. CHECK 检查表
11.3.2. ANALYZE 分析表
11.3.3. CHECKSUM
11.3.4. OPTIMIZE 优化表
11.3.5. REPAIR 修复
11.4. SHOW COMMAND
11.4.1. 查看版本
11.4.2. status
11.4.2.1. show status
11.4.2.2. show master status
11.4.2.3. show slave status
11.4.2.4. show plugins
11.4.3. show processlist
11.4.4. variables
11.4.4.1. wait_timeout
11.4.4.2. table_lock_wait_timeout
11.4.4.3. low_priority_updates
11.4.4.4. character_set
11.4.4.5. datadir
11.4.4.6. storage_engine
11.4.4.7. timeout
11.4.5. binary

11.1. User Account Management

11.1.1. Create User

CREATE USER user [IDENTIFIED BY [PASSWORD] 'password']
    [, user [IDENTIFIED BY [PASSWORD] 'password']] ...

CREATE USER 'test'@'xxx.xxx.xxx.xxx' IDENTIFIED BY  'your_password';

add a new user by grant

GRANT ALL PRIVILEGES ON opencart.* TO 'neo'@'localhost' IDENTIFIED BY 'chen' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'neo'@'localhost' IDENTIFIED BY 'chen' WITH GRANT OPTION;

FLUSH PRIVILEGES;

11.1.2. Drop User

DROP USER user [, user] ...

mysql> drop user admin@'localhost';
Query OK, 0 rows affected (0.00 sec)

mysql> drop user admin@'127.0.0.1';
Query OK, 0 rows affected (0.00 sec)

11.1.3. Rename User

RENAME USER old_user TO new_user [, old_user TO new_user] ...

11.1.4. SET PASSWORD

SET PASSWORD FOR 'bob'@'%.loc.gov' = PASSWORD('newpass');

SET PASSWORD FOR 'root'@'%' = PASSWORD('co2uqAMAho1aSOS62146Xoci6ogu4I');

CREATE USER 'root'@'192.168.1.%' IDENTIFIED BY 'password';

11.1.5. Reset root password

忘记root密码是使用

shell>mysqld_safe --skip-grant-tables &
shell>mysqladmin -u root flush-privileges password "newpassword"


11.2. Access Privilege System

global privileges
OR (database privileges AND host privileges)
OR table privileges
OR column privileges
OR routine privileges

Table 12.1. Permissible Privileges for GRANT and REVOKE

Privilege	Meaning
ALL [PRIVILEGES]	Grant all privileges at specified access level except GRANT OPTION
ALTER	Enable use of ALTER TABLE
ALTER ROUTINE	Enable stored routines to be altered or dropped
CREATE	Enable database and table creation
CREATE ROUTINE	Enable stored routine creation
CREATE TABLESPACE	Enable tablespaces and log file groups to be created, altered, or dropped
CREATE TEMPORARY TABLES	Enable use of CREATE TEMPORARY TABLE
CREATE USER	Enable use of CREATE USER, DROP USER, RENAME USER, and REVOKE ALL PRIVILEGES
CREATE VIEW	Enable views to be created or altered
DELETE	Enable use of DELETE
DROP	Enable databases, tables, and views to be dropped
EVENT	Enable use of events for the Event Scheduler
EXECUTE	Enable the user to execute stored routines
FILE	Enable the user to cause the server to read or write files
GRANT OPTION	Enable privileges to be granted to or removed from other accounts
INDEX	Enable indexes to be created or dropped
INSERT	Enable use of INSERT
LOCK TABLES	Enable use of LOCK TABLES on tables for which you have the SELECT privilege
PROCESS	Enable the user to see all processes with SHOW PROCESSLIST
PROXY	Enable user proxying
REFERENCES	Not implemented
RELOAD	Enable use of FLUSH operations
REPLICATION CLIENT	Enable the user to ask where master or slave servers are
REPLICATION SLAVE	Enable replication slaves to read binary log events from the master
SELECT	Enable use of SELECT
SHOW DATABASES	Enable SHOW DATABASES to show all databases
SHOW VIEW	Enable use of SHOW CREATE VIEW
SHUTDOWN	Enable use of mysqladmin shutdown
SUPER	Enable use of other administrative operations such as CHANGE MASTER TO, KILL, PURGE BINARY LOGS, SET GLOBAL, and mysqladmin debug command
TRIGGER	Enable trigger operations
UPDATE	Enable use of UPDATE
USAGE	Synonym for “no privileges”
http://dev.mysql.com/doc/refman/5.5/en/grant.html#grant-table-privileges

11.2.1. SHOW GRANTS

mysql> SHOW GRANTS FOR 'root'@'localhost';
+---------------------------------------------------------------------+
| Grants for root@localhost                                           |
+---------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION |
+---------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> show grants;
+---------------------------------------------------------------------------------+
| Grants for root@localhost                                           |
+---------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION |
+---------------------------------------------------------------------------------+
1 row in set (0.00 sec)

11.2.2. show privileges

mysql> show privileges;
+-------------------------+---------------------------------------+-------------------------------------------------------+
| Privilege               | Context                               | Comment                                               |
+-------------------------+---------------------------------------+-------------------------------------------------------+
| Alter                   | Tables                                | To alter the table                                    |
| Alter routine           | Functions,Procedures                  | To alter or drop stored functions/procedures          |
| Create                  | Databases,Tables,Indexes              | To create new databases and tables                    |
| Create routine          | Databases                             | To use CREATE FUNCTION/PROCEDURE                      |
| Create temporary tables | Databases                             | To use CREATE TEMPORARY TABLE                         |
| Create view             | Tables                                | To create new views                                   |
| Create user             | Server Admin                          | To create new users                                   |
| Delete                  | Tables                                | To delete existing rows                               |
| Drop                    | Databases,Tables                      | To drop databases, tables, and views                  |
| Event                   | Server Admin                          | To create, alter, drop and execute events             |
| Execute                 | Functions,Procedures                  | To execute stored routines                            |
| File                    | File access on server                 | To read and write files on the server                 |
| Grant option            | Databases,Tables,Functions,Procedures | To give to other users those privileges you possess   |
| Index                   | Tables                                | To create or drop indexes                             |
| Insert                  | Tables                                | To insert data into tables                            |
| Lock tables             | Databases                             | To use LOCK TABLES (together with SELECT privilege)   |
| Process                 | Server Admin                          | To view the plain text of currently executing queries |
| Proxy                   | Server Admin                          | To make proxy user possible                           |
| References              | Databases,Tables                      | To have references on tables                          |
| Reload                  | Server Admin                          | To reload or refresh tables, logs and privileges      |
| Replication client      | Server Admin                          | To ask where the slave or master servers are          |
| Replication slave       | Server Admin                          | To read binary log events from the master             |
| Select                  | Tables                                | To retrieve rows from table                           |
| Show databases          | Server Admin                          | To see all databases with SHOW DATABASES              |
| Show view               | Tables                                | To see views with SHOW CREATE VIEW                    |
| Shutdown                | Server Admin                          | To shut down the server                               |
| Super                   | Server Admin                          | To use KILL thread, SET GLOBAL, CHANGE MASTER, etc.   |
| Trigger                 | Tables                                | To use triggers                                       |
| Create tablespace       | Server Admin                          | To create/alter/drop tablespaces                      |
| Update                  | Tables                                | To update existing rows                               |
| Usage                   | Server Admin                          | No privileges - allow connect only                    |
+-------------------------+---------------------------------------+-------------------------------------------------------+
31 rows in set (0.00 sec)

11.2.3. Grant privileges

Global privileges

GRANT ALL ON *.* TO 'someuser'@'somehost';
GRANT SELECT, INSERT ON *.* TO 'someuser'@'somehost';

Database privileges

GRANT ALL ON mydb.* TO 'someuser'@'somehost';
GRANT SELECT, INSERT ON mydb.* TO 'someuser'@'somehost';

Table privileges

GRANT ALL ON mydb.mytbl TO 'someuser'@'somehost';
GRANT SELECT, INSERT ON mydb.mytbl TO 'someuser'@'somehost';

Column privileges

GRANT SELECT (col1), INSERT (col1,col2) ON mydb.mytbl TO 'someuser'@'somehost';

Routine privileges

GRANT CREATE ROUTINE ON mydb.* TO 'someuser'@'somehost';
GRANT EXECUTE ON PROCEDURE mydb.myproc TO 'someuser'@'somehost';

11.2.4. Revoke privileges

REVOKE
    priv_type [(column_list)]
      [, priv_type [(column_list)]] ...
    ON [object_type] priv_level
    FROM user [, user] ...

REVOKE ALL PRIVILEGES, GRANT OPTION
    FROM user [, user] ...

11.2.5. Show Privileges

mysql> select * from user where user = 'neo'\G
*************************** 1. row ***************************
                 Host: 192.168.0.5
                 User: neo
             Password: *7564B7B0A062C9523700601CBA1DCE1F861D6270
          Select_priv: Y
          Insert_priv: Y
          Update_priv: Y
          Delete_priv: Y
          Create_priv: Y
            Drop_priv: Y
          Reload_priv: Y
        Shutdown_priv: Y
         Process_priv: Y
            File_priv: Y
           Grant_priv: N
      References_priv: Y
           Index_priv: Y
           Alter_priv: Y
         Show_db_priv: Y
           Super_priv: Y
Create_tmp_table_priv: Y
     Lock_tables_priv: Y
         Execute_priv: Y
      Repl_slave_priv: Y
     Repl_client_priv: Y
     Create_view_priv: Y
       Show_view_priv: Y
  Create_routine_priv: Y
   Alter_routine_priv: Y
     Create_user_priv: Y
           Event_priv: Y
         Trigger_priv: Y
             ssl_type:
           ssl_cipher:
          x509_issuer:
         x509_subject:
        max_questions: 0
          max_updates: 0
      max_connections: 0
 max_user_connections: 0
1 row in set (0.00 sec)

mysql>

11.2.6. MAX_QUERIES_PER_HOUR/MAX_UPDATES_PER_HOUR

GRANT USAGE ON *.* TO ...
  WITH MAX_QUERIES_PER_HOUR 500 MAX_UPDATES_PER_HOUR 100;


11.3. Maintenance 数据库维护

11.3.1. CHECK 检查表

CHECK TABLE `dbname`.`actions`;
CHECK TABLE `dbname`.`actions` QUICK FAST MEDIUM EXTENDED CHANGED;

11.3.2. ANALYZE 分析表

ANALYZE TABLE `dbname`.`actions`;

11.3.3. CHECKSUM

CHECKSUM TABLE `dbname`.`actions` QUICK;

11.3.4. OPTIMIZE 优化表

OPTIMIZE TABLE `dbname`.`actions`;

11.3.5. REPAIR 修复

REPAIR TABLE `dbname`.`members`;
SHOW TABLE STATUS LIKE 'members';


11.4. SHOW COMMAND

SHOW DATABASES;
SHOW TABLE STATUS FROM `db`;
SHOW FUNCTION STATUS WHERE `Db`='db';
SHOW PROCEDURE STATUS WHERE `Db`='db';
SHOW TRIGGERS FROM `db`;

Show Global Status;
Show global variables;
Show full processlist;

11.4.1. 查看版本

Server

mysql> select version();
+-----------+
| version() |
+-----------+
| 5.0.77    |
+-----------+
1 row in set (0.00 sec)

mysql> status;
--------------
mysql  Ver 14.12 Distrib 5.0.77, for redhat-linux-gnu (x86_64) using readline 5.1

Connection id:          1533
Current database:
Current user:           root@localhost
SSL:                    Not in use
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server version:         5.0.77 Source distribution
Protocol version:       10
Connection:             Localhost via UNIX socket
Server characterset:    latin1
Db     characterset:    latin1
Client characterset:    latin1
Conn.  characterset:    latin1
UNIX socket:            /var/lib/mysql/mysql.sock
Uptime:                 1 day 21 hours 40 min 52 sec

Threads: 1  Questions: 22172  Slow queries: 0  Opens: 3130  Flush tables: 1  Open tables: 64  Queries per second avg: 0.135
--------------

Client

[root@development ~]# mysql -V
mysql  Ver 14.12 Distrib 5.0.77, for redhat-linux-gnu (x86_64) using readline 5.1

11.4.2. status

mysql> show status;

11.4.2.1. show status

数据库性能状态
(1)QPS(每秒Query量)
QPS = Questions(or Queries) / seconds
mysql > show /*50000 global */ status like 'Question';

(2)TPS(每秒事务量)
TPS = (Com_commit + Com_rollback) / seconds
mysql > show status like 'Com_commit';
mysql > show status like 'Com_rollback';

(3)key Buffer 命中率
key_buffer_read_hits = (1-key_reads / key_read_requests) * 100%
key_buffer_write_hits = (1-key_writes / key_write_requests) * 100%

mysql> show status like 'Key%';

(4)InnoDB Buffer命中率
innodb_buffer_read_hits = (1 - innodb_buffer_pool_reads / innodb_buffer_pool_read_requests) * 100%

mysql> show status like 'innodb_buffer_pool_read%';

(5)Query Cache命中率
Query_cache_hits = (Qcahce_hits / (Qcache_hits + Qcache_inserts )) * 100%;

mysql> show status like 'Qcache%';
(6)Table Cache状态量
mysql> show status like 'open%';

(7)Thread Cache 命中率
Thread_cache_hits = (1 - Threads_created / connections ) * 100%

mysql> show status like 'Thread%';

mysql> show status like 'Connections';

(8)锁定状态
mysql> show status like '%lock%';

(9)复制延时量
mysql > show slave status

(10) Tmp Table 状况(临时表状况)
mysql > show status like 'Create_tmp%';
(11) Binlog Cache 使用状况
mysql > show status like 'Binlog_cache%';

(12) Innodb_log_waits 量
mysql > show status like 'innodb_log_waits';

11.4.2.2. show master status

mysql> show master status;
+---------------------+-----------+--------------+------------------+
| File                | Position  | Binlog_Do_DB | Binlog_Ignore_DB |
+---------------------+-----------+--------------+------------------+
| DBMaster-bin.000018 | 409468882 | example      |                  |
+---------------------+-----------+--------------+------------------+
1 row in set (0.00 sec)

mysql>

11.4.2.3. show slave status

mysql> show slave status/G

得到的列表会有类似下面的数据：

File: mysql-bin.000001
Position: 1374
Binlog_Do_DB: test
Binlog_Ignore_DB: mysql

Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Slave_SQL_Running: Yes

11.4.2.4. show plugins

mysql> SHOW PLUGINS;
+------------+----------+----------------+---------+---------+
| Name       | Status   | Type           | Library | License |
+------------+----------+----------------+---------+---------+
| binlog     | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| partition  | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| ARCHIVE    | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| BLACKHOLE  | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| CSV        | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| FEDERATED  | DISABLED | STORAGE ENGINE | NULL    | GPL     |
| MEMORY     | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| InnoDB     | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| MyISAM     | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
| MRG_MYISAM | ACTIVE   | STORAGE ENGINE | NULL    | GPL     |
+------------+----------+----------------+---------+---------+
10 rows in set (0.00 sec)

11.4.3. show processlist

命令： show processlist;
如果是root帐号，你能看到所有用户的当前连接。如果是其它普通帐号，只能看到自己占用的连接。
show processlist;只列出前100条，如果想全列出请使用show full processlist;
mysql> show processlist;

命令： show status;

Aborted_clients 由于客户没有正确关闭连接已经死掉，已经放弃的连接数量。
Aborted_connects 尝试已经失败的MySQL服务器的连接的次数。
Connections 试图连接MySQL服务器的次数。
Created_tmp_tables 当执行语句时，已经被创造了的隐含临时表的数量。
Delayed_insert_threads 正在使用的延迟插入处理器线程的数量。
Delayed_writes 用INSERT DELAYED写入的行数。
Delayed_errors 用INSERT DELAYED写入的发生某些错误(可能重复键值)的行数。
Flush_commands 执行FLUSH命令的次数。
Handler_delete 请求从一张表中删除行的次数。
Handler_read_first 请求读入表中第一行的次数。
Handler_read_key 请求数字基于键读行。
Handler_read_next 请求读入基于一个键的一行的次数。
Handler_read_rnd 请求读入基于一个固定位置的一行的次数。
Handler_update 请求更新表中一行的次数。
Handler_write 请求向表中插入一行的次数。
Key_blocks_used 用于关键字缓存的块的数量。
Key_read_requests 请求从缓存读入一个键值的次数。
Key_reads 从磁盘物理读入一个键值的次数。
Key_write_requests 请求将一个关键字块写入缓存次数。
Key_writes 将一个键值块物理写入磁盘的次数。
Max_used_connections 同时使用的连接的最大数目。
Not_flushed_key_blocks 在键缓存中已经改变但是还没被清空到磁盘上的键块。
Not_flushed_delayed_rows 在INSERT DELAY队列中等待写入的行的数量。
Open_tables 打开表的数量。
Open_files 打开文件的数量。
Open_streams 打开流的数量(主要用于日志记载）
Opened_tables 已经打开的表的数量。
Questions 发往服务器的查询的数量。
Slow_queries 要花超过long_query_time时间的查询数量。
Threads_connected 当前打开的连接的数量。
Threads_running 不在睡眠的线程数量。
Uptime 服务器工作了多少秒。

11.4.4. variables

11.4.4.1. wait_timeout

mysql> show global variables like 'wait_timeout';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| wait_timeout               | 10    |
+----------------------------+-------+

mysql> use mysql;
Database changed

mysql> set wait_timeout=10;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables  like '%wait_timeout%';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| innodb_lock_wait_timeout | 50    |
| table_lock_wait_timeout  | 50    |
| wait_timeout             | 10    |
+--------------------------+-------+
3 rows in set (0.00 sec)

11.4.4.2. table_lock_wait_timeout

mysql> set GLOBAL table_lock_wait_timeout=10;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables  like '%table_lock_wait_timeout%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| table_lock_wait_timeout | 10    |
+-------------------------+-------+
1 row in set (0.00 sec)

11.4.4.3. low_priority_updates

mysql> use mysql
Database changed

mysql> SET LOW_PRIORITY_UPDATES=1;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables  like '%priority%';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| low_priority_updates     | ON    |
| sql_low_priority_updates | ON    |
+--------------------------+-------+
2 rows in set (0.00 sec)

11.4.4.4. character_set

mysql> show variables like 'character%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

11.4.4.5. datadir

 SHOW VARIABLES LIKE 'datadir';

mysql> SHOW VARIABLES LIKE 'datadir';
+---------------+-------------------------+
| Variable_name | Value                   |
+---------------+-------------------------+
| datadir       | /var/lib/mysql/         |
+---------------+-------------------------+
1 row in set (0.00 sec)

11.4.4.6. storage_engine

mysql> show variables like '%storage_engine%';
+------------------------+--------+
| Variable_name          | Value  |
+------------------------+--------+
| default_storage_engine | InnoDB |
| storage_engine         | InnoDB |
+------------------------+--------+
2 rows in set (0.00 sec)

11.4.4.7. timeout

show variables like "%timeout%";

11.4.5. binary

mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |     19544 |
| mysql-bin.000002 |    974751 |
| mysql-bin.000003 |       107 |
| mysql-bin.000004 |   3976040 |
| mysql-bin.000005 |       126 |
| mysql-bin.000006 |    350063 |
| mysql-bin.000007 |      6826 |
| mysql-bin.000008 |   3879494 |
| mysql-bin.000009 |       126 |
| mysql-bin.000010 |       494 |
| mysql-bin.000011 |  17286686 |
| mysql-bin.000012 |  15003942 |
| mysql-bin.000013 |   1709321 |
+------------------+-----------+
13 rows in set (0.00 sec)


第 12 章 DDL - Data Definition Language

目录

12.1. Database
12.1.1. create
12.1.2. drop
12.1.3. Alter
12.1.4. Rename
12.1.5. CHARACTER
12.1.6. show create database
12.2. Table
12.2.1. create table ... select
12.2.2. modifiy table
12.2.3. TEMPORARY Table
12.2.4. Collate
12.2.5. CHARACTER
12.2.6. DEFAULT
12.2.7. KEY
12.2.7.1. PRIMARY KEY
12.2.8. COMMENT
12.3. Index
12.3.1. SHOW INDEX
12.3.2. CREATE INDEX
12.3.3. DROP INDEX
12.3.4. rebuild
12.4. 外键(Foreign Key)
12.4.1. FOREIGN KEY (RESTRICT)
12.5. View
12.6. Trigger
12.6.1. create trigger
12.6.2. drop trigger
12.6.3. show triggers
12.6.4. EXAMPLE
12.7. EVENT
12.7.1. 启用 EVENT
12.7.2. 创建 EVENT
12.7.3. 禁用/启用
12.7.4. show events
12.8. Partitioning
12.8.1. RANGE
12.8.2. LIST
12.8.3. HASH
12.8.3.1. LINEAR HASH
12.8.4. KEY分区
12.8.5. 分区管理
12.8.5.1. 新增分区
12.8.5.2. 删除分区
12.8.5.3. 重建分区
12.8.5.4. 分区维护
12.8.6. EXPLAIN PARTITIONS
12.8.7. SHOW CREATE TABLE
12.8.8. INFORMATION_SCHEMA.partitions 表
12.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
12.9. Engine
12.9.1. FEDERATED
12.9.2. BLACKHOLE

12.1. Database

12.1.1. create

Creating a UTF-8 database

CREATE DATABASE db_name DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

Create a UTF-8 database with binary UTF-8 collation.

CREATE DATABASE dbname CHARACTER SET utf8 COLLATE utf8_bin;

12.1.2. drop

DROP DATABASE db_name;

12.1.3. Alter

ALTER DATABASE dbname DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

12.1.4. Rename

RENAME {DATABASE | SCHEMA} db_name TO new_db_name;

before 5.0 version

[neo@development ~]$ mysqldump -uroot -pchen db_old | mysql -uroot -pchen db_new

12.1.5. CHARACTER

ALTER DATABASE <database_name> CHARACTER SET utf8;

12.1.6. show create database

mysql> show create database dbname;
+----------+-------------------------------------------------------------------+
| Database | Create Database                                                   |
+----------+-------------------------------------------------------------------+
| dbname   | CREATE DATABASE `dbname` /*!40100 DEFAULT CHARACTER SET utf8 */   |
+----------+-------------------------------------------------------------------+
1 row in set (0.00 sec)


12.2. Table

12.2.1. create table ... select

创建空表

create table admin_user_history select * from admin_user where 1 <> 1;

创建有数据的表

create table admin_user_history select * from admin_user;

12.2.2. modifiy table

modifiy table

ALTER TABLE ecs_users add user_picture varchar(255);

12.2.3. TEMPORARY Table

临时表将在你连接期间存在。一旦断开时将自动删除表并释放所用的空间。你在连接期间删除该表也同样释放空间。

CREATE TEMPORARY TABLE tmp_table (
    key VARCHAR(10) NOT NULL,
    value INTEGER NOT NULL
)

声明临时表是一个HEAP表，允许你指定在内存中创建它

CREATE TEMPORARY TABLE tmp_mem_table (
    key VARCHAR(10) NOT NULL,
    value INTEGER NOT NULL
) TYPE = HEAP

12.2.4. Collate

ALTER TABLE `tmp_cats`  COLLATE='utf8_general_ci',  CONVERT TO CHARSET utf8;

12.2.5. CHARACTER

ALTER TABLE <table_name> CONVERT TO CHARACTER SET utf8;

12.2.6. DEFAULT

更新时间

`mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更改时间',

CREATE TABLE `bank_account` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '自增唯一ID',
    `bank_name` VARCHAR(255) NOT NULL DEFAULT '0' COMMENT '银行名字(Bank Name)',
    `name` VARCHAR(50) NOT NULL DEFAULT '0' COMMENT '帐号名称(Name on account)',
    `account_number` VARCHAR(50) NOT NULL DEFAULT '0' COMMENT '银行帐号(Account Number)',
    `branch_location` VARCHAR(255) NOT NULL DEFAULT '0' COMMENT '支行位置(Branch Location)',
    `description` VARCHAR(255) NOT NULL DEFAULT '0' COMMENT '银行描述',
    `status` ENUM('Y','N') NOT NULL DEFAULT 'N' COMMENT '银行帐号状态',
    `ctime` DATETIME NOT NULL COMMENT '创建时间',
    `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更改时间',
    PRIMARY KEY (`id`)
)
COMMENT='银行帐号'
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=2;

12.2.7. KEY

12.2.7.1. PRIMARY KEY

一般主键定义

PRIMARY KEY (`id`),

复合主键

PRIMARY KEY (`id`, `user_id`),

12.2.8. COMMENT

ALTER TABLE `neo`.`stuff` COMMENT = '用户表' ;
ALTER TABLE `neo`.`stuff` CHANGE COLUMN `name` `name` VARCHAR(50) NULL DEFAULT NULL COMMENT '姓名'  ;
ALTER TABLE `neo`.`stuff` CHANGE COLUMN `password` `password` VARCHAR(50) NULL DEFAULT NULL COMMENT '用户密码' ;

CREATE TABLE `stuff` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(50) DEFAULT NULL COMMENT ''姓名'',
  `password` varchar(50) DEFAULT NULL COMMENT ''用户密码'',
  `created` date NOT NULL DEFAULT ''0000-00-00'',
  PRIMARY KEY (`id`,`created`)
) ENGINE=MyISAM AUTO_INCREMENT=5 DEFAULT CHARSET=latin1 COMMENT=''用户表''
/*!50100 PARTITION BY HASH (year(created))
PARTITIONS 10 */


12.3. Index

12.3.1. SHOW INDEX

SHOW INDEX FROM tbl_name

垂直显示

SHOW INDEX FROM tbl_name\G

12.3.2. CREATE INDEX

CREATE INDEX index_name
ON table_name (column_name)

CREATE UNIQUE INDEX

CREATE UNIQUE INDEX index_name
ON table_name (column_name)

12.3.3. DROP INDEX

DROP INDEX index_name ON tbl_name

12.3.4. rebuild

SHOW INDEX FROM tbl_name
alter index IND_PK rebuild;


12.4. 外键(Foreign Key)

ON DELETE, ON UPDATE 事件触发限制，可选参数： RESTRICT | CASCADE | SET NULL | NO ACTION

  1. RESTRICT（限制外表中的外键改动）

  2. CASCADE（跟随外键改动）

  3. SET NULL（设空值）

  4. SET DEFAULT（设默认值）

  5. NO ACTION（无动作，默认的）

12.4.1. FOREIGN KEY (RESTRICT)

CREATE TABLE `bank_account_group_has_bank_account` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `bank_account_group_id` INT(10) UNSIGNED NOT NULL DEFAULT '0',
    `bank_account_id` INT(10) UNSIGNED NOT NULL DEFAULT '0',
    PRIMARY KEY (`id`),
    INDEX `FK_bank_account_group_has_bank_account_bank_account` (`bank_account_id`),
    INDEX `FK_bank_account_group_has_bank_account_bank_account_group` (`bank_account_group_id`),
    CONSTRAINT `FK_bank_account_group_has_bank_account_bank_account` FOREIGN KEY (`bank_account_id`) REFERENCES `bank_account` (`id`),
    CONSTRAINT `FK_bank_account_group_has_bank_account_bank_account_group` FOREIGN KEY (`bank_account_group_id`) REFERENCES `bank_account_group` (`id`)
)
COMMENT='bank_account_group 与 bank_account 的 N:M 关系'
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=35;


12.5. View

CREATE VIEW view_name AS
SELECT column_name(s)
FROM table_name
WHERE condition

update view

SQL CREATE OR REPLACE VIEW Syntax
CREATE OR REPLACE VIEW view_name AS
SELECT column_name(s)
FROM table_name
WHERE condition


12.6. Trigger

12.6.1. create trigger

history

CREATE TABLE user_history SELECT * FROM user WHERE 1 <> 1

DELIMITER //
CREATE TRIGGER user_history BEFORE update ON user FOR EACH ROW
BEGIN
insert into user_history SELECT * FROM user WHERE id = OLD.id;
END; //
DELIMITER ;

12.6.2. drop trigger

DROP TRIGGER admin_user_history;

DELIMITER //
CREATE TRIGGER admin_user_history BEFORE update ON admin_user FOR EACH ROW
BEGIN
insert into admin_user_history SELECT * FROM admin_user WHERE user_id = OLD.user_id;
END; //
DELIMITER;

12.6.3. show triggers

show triggers;

12.6.4. EXAMPLE

例 12.1. BEFORE/AFTER

DROP TRIGGER MY_TEST_MONITOR;
DELIMITER //
CREATE TRIGGER MY_TEST_MONITOR BEFORE insert ON MY_TEST FOR EACH ROW
BEGIN
    INSERT INTO MY_TEST_MONITOR SELECT * FROM MY_TEST WHERE TICKET = NEW.TICKET;
END; //
DELIMITER;

DROP TRIGGER MY_TEST_MONITOR;
DELIMITER //
CREATE TRIGGER MY_TEST_MONITOR AFTER insert ON MY_TEST FOR EACH ROW
BEGIN
    INSERT INTO MY_TEST_MONITOR SELECT * FROM MY_TEST WHERE TICKET = NEW.TICKET;
END; //
DELIMITER;

例 12.2. uuid()

delimiter $$

CREATE TABLE `member` (
  `uuid` char(36) NOT NULL,
  `username` varchar(20) DEFAULT NULL,
  `password` varchar(32) DEFAULT NULL,
  PRIMARY KEY (`uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8$$

CREATE
DEFINER=`root`@`%`
TRIGGER `test`.`member_before_insert`
BEFORE INSERT ON `test`.`member`
FOR EACH ROW
SET new.uuid = uuid()
$$


12.7. EVENT

12.7.1. 启用 EVENT

set GLOBAL event_scheduler=ON;

my.cnf 配置

event_scheduler=on

查看状态

mysql> select @@GLOBAL.event_scheduler;
+--------------------------+
| @@GLOBAL.event_scheduler |
+--------------------------+
| ON                       |
+--------------------------+
1 row in set (0.00 sec)

mysql> SHOW VARIABLES LIKE 'event_scheduler';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| event_scheduler | ON    |
+-----------------+-------+
1 row in set (0.01 sec)

12.7.2. 创建 EVENT

DROP EVENT IF EXISTS `captcha`;
DELIMITER //
CREATE DEFINER=`neo`@`%` EVENT `captcha` ON SCHEDULE EVERY 5 MINUTE STARTS '2013-07-08 16:27:03' ON COMPLETION PRESERVE ENABLE DO BEGIN
    delete from captcha where ctime < DATE_ADD(now(), INTERVAL -5 MINUTE);
END//
DELIMITER ;

12.7.3. 禁用/启用

ALTER EVENT captcha DISABLE;

ALTER EVENT captcha ENABLE;

12.7.4. show events

mysql> show events;
+--------+-------------+---------+-----------+-----------+------------+----------------+----------------+---------------------+------+---------+------------+----------------------+----------------------+--------------------+
| Db     | Name        | Definer | Time zone | Type      | Execute at | Interval value | Interval field | Starts              | Ends | Status  | Originator | character_set_client | collation_connection | Database Collation |
+--------+-------------+---------+-----------+-----------+------------+----------------+----------------+---------------------+------+---------+------------+----------------------+----------------------+--------------------+
| hx9999 | captcha     | neo@%   | SYSTEM    | RECURRING | NULL       | 5              | MINUTE         | 2013-07-08 16:27:03 | NULL | ENABLED |          1 | utf8                 | utf8_general_ci      | utf8_general_ci    |
| hx9999 | sms_ips_log | neo@%   | SYSTEM    | RECURRING | NULL       | '0 5'          | DAY_HOUR       | 2013-07-09 14:39:51 | NULL | ENABLED |          1 | utf8                 | utf8_general_ci      | utf8_general_ci    |
+--------+-------------+---------+-----------+-----------+------------+----------------+----------------+---------------------+------+---------+------------+----------------------+----------------------+--------------------+
2 rows in set (0.00 sec)

mysql> show events \G;
*************************** 1. row ***************************
                  Db: hx9999
                Name: captcha
             Definer: neo@%
           Time zone: SYSTEM
                Type: RECURRING
          Execute at: NULL
      Interval value: 5
      Interval field: MINUTE
              Starts: 2013-07-08 16:27:03
                Ends: NULL
              Status: ENABLED
          Originator: 1
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
*************************** 2. row ***************************
                  Db: hx9999
                Name: sms_ips_log
             Definer: neo@%
           Time zone: SYSTEM
                Type: RECURRING
          Execute at: NULL
      Interval value: '0 5'
      Interval field: DAY_HOUR
              Starts: 2013-07-09 14:39:51
                Ends: NULL
              Status: ENABLED
          Originator: 1
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
2 rows in set (0.00 sec)

ERROR:
No query specified


12.8. Partitioning

mysql> SHOW VARIABLES LIKE '%partition%';

+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| have_partitioning | YES   |
+-------------------+-------+
1 row in set (0.00 sec)

12.8.1. RANGE

18.5.1. Partitioning Keys, Primary Keys, and Unique Keys
This section discusses the relationship of partitioning keys with primary keys and unique keys. The rule governing this relationship can be expressed as follows: All columns used in the partitioning expression for a partitioned table must be part of every unique key that the table may have.

In other words, every unique key on the table must use every column in the table's partitioning expression. (This also includes the table's primary key, since it is by definition a unique key. This particular case is discussed later in this section.) For example, each of the following table creation statements is invalid:

SQL code:
mysql> create table tx (
    ->     id int not null ,
    ->     info_time date,
    ->     primary key(id,info_time)
    -> )
    -> PARTITION BY RANGE(info_time div 100)
    -> (
    ->     PARTITION p_2008_11 VALUES LESS THAN (200812),
    ->     PARTITION p_2008_12 VALUES LESS THAN (200901),
    ->     PARTITION p_2009_01 VALUES LESS THAN (200902),
    ->     PARTITION p_2009_02 VALUES LESS THAN (200903),
    ->     PARTITION p_2009_03 VALUES LESS THAN (200904),
    ->     PARTITION p_2009_04 VALUES LESS THAN (200905),
    ->     PARTITION p_catch_all VALUES LESS THAN MAXVALUE
    -> );
Query OK, 0 rows affected (0.17 sec)

mysql>

CREATE TABLE t1 (
    year_col  INT,
    some_data INT
)
PARTITION BY RANGE (year_col) (
    PARTITION p0 VALUES LESS THAN (1991),
    PARTITION p1 VALUES LESS THAN (1995),
    PARTITION p2 VALUES LESS THAN (1999),
    PARTITION p3 VALUES LESS THAN (2002),
    PARTITION p4 VALUES LESS THAN (2006),
    PARTITION p5 VALUES LESS THAN MAXVALUE
);

e.g.2

CREATE TABLE rc (
    a INT NOT NULL,
    b INT NOT NULL
)
PARTITION BY RANGE COLUMNS(a,b) (
    PARTITION p0 VALUES LESS THAN (10,5),
    PARTITION p1 VALUES LESS THAN (20,10),
    PARTITION p2 VALUES LESS THAN (MAXVALUE,15),
    PARTITION p3 VALUES LESS THAN (MAXVALUE,MAXVALUE)
);

CREATE TABLE part_tab
(
    c1 int default NULL,
    c2 varchar(30) default NULL,
    c3 date default NULL

) engine=myisam
PARTITION BY RANGE (year(c3)) (
      PARTITION p0 VALUES LESS THAN (2000) ,
      PARTITION p1 VALUES LESS THAN (2001) ,
      PARTITION p2 VALUES LESS THAN (2002) ,
      PARTITION p3 VALUES LESS THAN (2003) ,
      PARTITION p4 VALUES LESS THAN (2004) ,
      PARTITION p12 VALUES LESS THAN (2012),
      PARTITION p13 VALUES LESS THAN MAXVALUE
);

12.8.2. LIST

CREATE TABLE client_firms (
    id   INT,
    name VARCHAR(35)
)
PARTITION BY LIST (id) (
    PARTITION r0 VALUES IN (1, 5, 9, 13, 17, 21),
    PARTITION r1 VALUES IN (2, 6, 10, 14, 18, 22),
    PARTITION r2 VALUES IN (3, 7, 11, 15, 19, 23),
    PARTITION r3 VALUES IN (4, 8, 12, 16, 20, 24)
);

CREATE TABLE lc (
    a INT NULL,
    b INT NULL
)
PARTITION BY LIST COLUMNS(a,b) (
    PARTITION p0 VALUES IN( (0,0), (NULL,NULL) ),
    PARTITION p1 VALUES IN( (0,1), (0,2), (0,3), (1,1), (1,2) ),
    PARTITION p2 VALUES IN( (1,0), (2,0), (2,1), (3,0), (3,1) ),
    PARTITION p3 VALUES IN( (1,3), (2,2), (2,3), (3,2), (3,3) )
);

CREATE TABLE th (id INT, name VARCHAR(30), adate DATE)
PARTITION BY LIST(YEAR(adate))
(
  PARTITION p1999 VALUES IN (1995, 1999, 2003)
    DATA DIRECTORY = '/var/appdata/95/data'
    INDEX DIRECTORY = '/var/appdata/95/idx',
  PARTITION p2000 VALUES IN (1996, 2000, 2004)
    DATA DIRECTORY = '/var/appdata/96/data'
    INDEX DIRECTORY = '/var/appdata/96/idx',
  PARTITION p2001 VALUES IN (1997, 2001, 2005)
    DATA DIRECTORY = '/var/appdata/97/data'
    INDEX DIRECTORY = '/var/appdata/97/idx',
  PARTITION p2000 VALUES IN (1998, 2002, 2006)
    DATA DIRECTORY = '/var/appdata/98/data'
    INDEX DIRECTORY = '/var/appdata/98/idx'
);

12.8.3. HASH

CREATE TABLE `test` (
  `userid` int(10) unsigned NOT NULL auto_increment,
  `username` int(10) unsigned NOT NULL DEFAULT '0',
  `password` int(10) unsigned NOT NULL DEFAULT '0',

  primary key (`userid`),
  KEY `userid` (`username`)
) ENGINE=InnoDB
PARTITION BY HASH(userid)
PARTITIONS 8;

CREATE TABLE stuff (
    id INT AUTO_INCREMENT,
    name varchar(50),
    password varchar(50),
    created DATE,
    PRIMARY KEY (id, created)
)
PARTITION BY RANGE(year(created)) (
    PARTITION p0 VALUES LESS THAN (2010),
    PARTITION p1 VALUES LESS THAN (2012),
    PARTITION p2 VALUES LESS THAN MAXVALUE
);

更好的方法

CREATE TABLE stuff (
    id INT AUTO_INCREMENT,
    name varchar(50),
    password varchar(50),
    created DATE,
    PRIMARY KEY (id, created)
)
PARTITION BY HASH (year(created)) PARTITIONS 10;

我们演示一下

mysql> CREATE TABLE stuff (
    -> id INT AUTO_INCREMENT,
    -> name varchar(50),
    -> password varchar(50),
    -> created DATE,
    -> PRIMARY KEY (id, created)
    -> )
    -> PARTITION BY HASH (year(created)) PARTITIONS 10;
Query OK, 0 rows affected (0.08 sec)

mysql> insert into stuff (name,password,created) values('neo','test','2010-10-1');
Query OK, 1 row affected (0.06 sec)

mysql> insert into stuff (name,password,created) values('neo1','test','2012-2-1');
Query OK, 1 row affected (0.00 sec)

mysql> insert into stuff (name,password,created) values('neo2','test','2012-3-5');
Query OK, 1 row affected (0.00 sec)

mysql> insert into stuff (name,password,created) values('neo4','test','2011-1-5');
Query OK, 1 row affected (0.00 sec)

mysql> SELECT
    ->   partition_name part,
    ->   partition_expression expr,
    ->   partition_description descr,
    ->   table_rows
    -> FROM
    ->   INFORMATION_SCHEMA.partitions
    -> WHERE
    ->   TABLE_SCHEMA = schema()
    ->   AND TABLE_NAME='stuff';
+------+---------------+-------+------------+
| part | expr          | descr | table_rows |
+------+---------------+-------+------------+
| p0   | year(created) | NULL  |          1 |
| p1   | year(created) | NULL  |          1 |
| p2   | year(created) | NULL  |          2 |
| p3   | year(created) | NULL  |          0 |
| p4   | year(created) | NULL  |          0 |
| p5   | year(created) | NULL  |          0 |
| p6   | year(created) | NULL  |          0 |
| p7   | year(created) | NULL  |          0 |
| p8   | year(created) | NULL  |          0 |
| p9   | year(created) | NULL  |          0 |
+------+---------------+-------+------------+
10 rows in set (0.02 sec)

mysql> EXPLAIN PARTITIONS SELECT * FROM stuff WHERE created='2011-01-05'\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: stuff
   partitions: p1
         type: system
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 1
        Extra:
1 row in set (0.08 sec)

mysql> EXPLAIN PARTITIONS SELECT * FROM stuff WHERE created='2012-03-05'\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: stuff
   partitions: p2
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 2
        Extra: Using where
1 row in set (0.00 sec)

12.8.3.1. LINEAR HASH

CREATE TABLE employees (
    id INT NOT NULL,
    fname VARCHAR(30),
    lname VARCHAR(30),
    hired DATE NOT NULL DEFAULT '1970-01-01',
    separated DATE NOT NULL DEFAULT '9999-12-31',
    job_code INT,
    store_id INT
)
PARTITION BY LINEAR HASH( YEAR(hired) )
PARTITIONS 4;

12.8.4. KEY分区

按照KEY进行分区类似于按照HASH分区，除了HASH分区使用的用户定义的表达式，而KEY分区的 哈希函数是由MySQL 服务器提供。MySQL 簇（Cluster）使用函数MD5()来实现KEY分区；

CREATE TABLE tk (
    col1 INT NOT NULL,
    col2 CHAR(5),
    col3 DATE
)
PARTITION BY LINEAR KEY (col1)
PARTITIONS 3;

12.8.5. 分区管理

12.8.5.1. 新增分区

mysql 5.5+

为已经存在表添加分区

ALTER TABLE tbl_name  ADD PARTITION PARTITIONS 6;

新增 RANGE分区

ALTER TABLE category ADD PARTITION (PARTITION p4 VALUES IN (100,200,300,400)
                    DATA DIRECTORY = '/data/category'
                    INDEX DIRECTORY = '/data/category');

新增 LIST分区

CREATE TABLE expenses (
  expense_date DATE NOT NULL,
  category VARCHAR(30),
  amount DECIMAL (10,3)
);

ALTER TABLE expenses
PARTITION BY LIST COLUMNS (category)
(
  PARTITION p01 VALUES IN ( 'lodging', 'food'),
  PARTITION p02 VALUES IN ( 'flights', 'ground transportation'),
  PARTITION p03 VALUES IN ( 'leisure', 'customer entertainment'),
  PARTITION p04 VALUES IN ( 'communications'),
  PARTITION p05 VALUES IN ( 'fees')
);

新增 HASH分区

CREATE TABLE t1 (
    id INT,
    year_col INT
);

ALTER TABLE t1
    PARTITION BY HASH(id)
    PARTITIONS 8;

/* 在MySQL 5.1中*/
CREATE TABLE t2
(
  dt DATE
)
PARTITION BY RANGE (TO_DAYS(dt))
(
  PARTITION p01 VALUES LESS THAN (TO_DAYS('2007-01-01')),
  PARTITION p02 VALUES LESS THAN (TO_DAYS('2008-01-01')),
  PARTITION p03 VALUES LESS THAN (TO_DAYS('2009-01-01')),
  PARTITION p04 VALUES LESS THAN (MAXVALUE));

SHOW CREATE TABLE t2 \G
*************************** 1. row ***************************
       Table: t2
Create Table: CREATE TABLE `t2` (
  `dt` date DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/*!50100 PARTITION BY RANGE (TO_DAYS(dt))
(PARTITION p01 VALUES LESS THAN (733042) ENGINE = MyISAM,
 PARTITION p02 VALUES LESS THAN (733407) ENGINE = MyISAM,
 PARTITION p03 VALUES LESS THAN (733773) ENGINE = MyISAM,
 PARTITION p04 VALUES LESS THAN MAXVALUE ENGINE = MyISAM) */

 /*在MySQL 5.5中*/
CREATE TABLE t2
(
  dt DATE
)
PARTITION BY RANGE COLUMNS (dt)
(
  PARTITION p01 VALUES LESS THAN ('2007-01-01'),
  PARTITION p02 VALUES LESS THAN ('2008-01-01'),
  PARTITION p03 VALUES LESS THAN ('2009-01-01'),
  PARTITION p04 VALUES LESS THAN (MAXVALUE));

SHOW CREATE TABLE t2 \G
*************************** 1. row ***************************
       Table: t2
Create Table: CREATE TABLE `t2` (
  `dt` date DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/*!50500 PARTITION BY RANGE  COLUMNS(dt)
(PARTITION p01 VALUES LESS THAN ('2007-01-01') ENGINE = MyISAM,
 PARTITION p02 VALUES LESS THAN ('2008-01-01') ENGINE = MyISAM,
 PARTITION p03 VALUES LESS THAN ('2009-01-01') ENGINE = MyISAM,
 PARTITION p04 VALUES LESS THAN (MAXVALUE) ENGINE = MyISAM) */

12.8.5.2. 删除分区

删除分区 p0

ALERT TABLE users DROP PARTITION p0;

12.8.5.3. 重建分区

使用 REORGANIZE 重建分区。

RANGE 分区重建
ALTER TABLE users REORGANIZE PARTITION p0,p1 INTO (PARTITION p0 VALUES LESS THAN (6000000));

将原来的 p0,p1 分区合并起来，放到新的 p0 分区中。

LIST 分区重建
ALTER TABLE users REORGANIZE PARTITION p0,p1 INTO (PARTITION p0 VALUES IN(0,1,4,5,8,9,12,13));
将原来的 p0,p1 分区合并起来，放到新的 p0 分区中。

HASH/KEY 分区重建
ALTER TABLE users REORGANIZE PARTITION COALESCE PARTITION 2;
分区的数量改为2，
注意：在这里数量只能减少不能增加。想要增加可以用 ADD PARTITION 方法

调整HASH/KEY分区数量，将分区总数扩展到8个。

ALTER TABLE users ADD PARTITION PARTITIONS 8;

12.8.5.4. 分区维护

重建分区: 这和先删除保存在分区中的所有记录，然后重新插入它们，具有同样的效果。它可用于整理分区碎片。

示例：

ALTER TABLE t1 REBUILD PARTITION (p0, p1)；
·         优化分区：如果从分区中删除了大量的行，或者对一个带有可变长度的行（也就是说，有VARCHAR，BLOB，或TEXT类型的列）作了许多修改，可以使用“ALTER TABLE ... OPTIMIZE PARTITION”来收回没有使用的空间，并整理分区数据文件的碎片。

示例：

ALTER TABLE t1 OPTIMIZE PARTITION (p0, p1)；
在一个给定的分区表上使用“OPTIMIZE PARTITION”等同于在那个分区上运行CHECK PARTITION，ANALYZE PARTITION，和REPAIR PARTITION。

·         分析分区：读取并保存分区的键分布。

示例：

ALTER TABLE t1 ANALYZE PARTITION (p3)；
·         修补分区： 修补被破坏的分区。

示例：

ALTER TABLE t1 REPAIR PARTITION (p0,p1);
·         检查分区： 可以使用几乎与对非分区表使用CHECK TABLE 相同的方式检查分区。

示例：

ALTER TABLE trb3 CHECK PARTITION (p1)；

12.8.6. EXPLAIN PARTITIONS

EXPLAIN PARTITIONS

mysql> EXPLAIN PARTITIONS SELECT * FROM users\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: users
   partitions: p0,p1,p2,p3,p4,p5,p6
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 7
        Extra:
1 row in set (0.03 sec)

mysql> EXPLAIN PARTITIONS SELECT * FROM users WHERE id < 5\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: users
   partitions: p0,p1,p2,p3,p4,p5,p6
         type: range
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 4
          ref: NULL
         rows: 7
        Extra: Using where
1 row in set (0.00 sec)

12.8.7. SHOW CREATE TABLE

SHOW CREATE TABLE

mysql> SHOW CREATE TABLE users\G
*************************** 1. row ***************************
       Table: users
Create Table: CREATE TABLE `users` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(20) NOT NULL DEFAULT '',
  `birthday` datetime DEFAULT NULL,
  PRIMARY KEY (`id`,`username`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1
/*!50100 PARTITION BY KEY (id,username)
PARTITIONS 7 */
1 row in set (0.00 sec)

12.8.8. INFORMATION_SCHEMA.partitions 表

SELECT
  partition_name part,
  partition_expression expr,
  partition_description descr,
  table_rows
FROM
  INFORMATION_SCHEMA.partitions
WHERE
  TABLE_SCHEMA = schema()
  AND TABLE_NAME='employees';

select
  partition_name part,
  partition_expression expr,
  from_seconds(partition_description) descr,
  table_rows
FROM
INFORMATION_SCHEMA.partitions
WHERE
    TABLE_SCHEMA = 'test'
    AND TABLE_NAME='t2';

12.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function

http://dev.mysql.com/doc/refman/5.1/en/partitioning-limitations-partitioning-keys-unique-keys.html


12.9. Engine

mysql> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| FEDERATED          | NO      | Federated MySQL storage engine                                 | NULL         | NULL | NULL       |
| MRG_MYISAM         | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| MyISAM             | YES     | MyISAM storage engine                                          | NO           | NO   | NO         |
| BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| CSV                | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| ARCHIVE            | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| PERFORMANCE_SCHEMA | YES     | Performance Schema                                             | NO           | NO   | NO         |
| InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)

修改与切换引擎

ALTER TABLE `test` ENGINE=BLACKHOLE;
ALTER TABLE `test` ENGINE=InnoDB;

12.9.1. FEDERATED

启用 FEDERATED 引擎, 服务器环境 Ubuntu 13.04

$ sudo vim /etc/mysql/conf.d/federated.cnf
[mysqld]
federated

$ sudo service mysql restart

mysql> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| FEDERATED          | YES     | Federated MySQL storage engine                                 | NO           | NO   | NO         |
| MRG_MYISAM         | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| MyISAM             | YES     | MyISAM storage engine                                          | NO           | NO   | NO         |
| BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| CSV                | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| ARCHIVE            | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| PERFORMANCE_SCHEMA | YES     | Performance Schema                                             | NO           | NO   | NO         |
| InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)

A 服务器

CREATE TABLE `t1` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `name` VARCHAR(50) NOT NULL,
    `sex` ENUM('Y','N') NULL DEFAULT 'Y',
    `passwd` VARCHAR(50) NULL DEFAULT NULL,
    `ctime` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
    `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (`id`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=4;

B 服务器

DROP TABLE `users`;

CREATE TABLE `users` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `name` VARCHAR(50) NOT NULL,
    `sex` ENUM('Y','N') NULL DEFAULT 'Y',
    `ctime` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
    `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (`id`)
) ENGINE=FEDERATED connection = 'mysql://www:qwer123@192.168.2.1:3306/test/t1';

上面字段描述是你需要的字段，并非所有字段。这里屏蔽了passwd字段

提示

connection = 'mysql://用户名:密码@主机:端口/数据库/表名'

mysql> DROP TABLE `users`;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE TABLE `users` (
    -> `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    -> `name` VARCHAR(50) NOT NULL,
    -> `sex` ENUM('Y','N') NULL DEFAULT 'Y',
    -> `ctime` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
    -> `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    -> PRIMARY KEY (`id`)
    -> ) ENGINE=FEDERATED connection = 'mysql://www:qwer123@192.168.2.1:3306/test/t1';
Query OK, 0 rows affected (0.06 sec)

mysql>
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| users          |
+----------------+
1 row in set (0.00 sec)

mysql> desc users;
+-------+------------------+------+-----+---------------------+-----------------------------+
| Field | Type             | Null | Key | Default             | Extra                       |
+-------+------------------+------+-----+---------------------+-----------------------------+
| id    | int(10) unsigned | NO   | PRI | NULL                | auto_increment              |
| name  | varchar(50)      | NO   |     | NULL                |                             |
| sex   | enum('Y','N')    | YES  |     | Y                   |                             |
| ctime | timestamp        | NO   |     | 0000-00-00 00:00:00 |                             |
| mtime | timestamp        | NO   |     | CURRENT_TIMESTAMP   | on update CURRENT_TIMESTAMP |
+-------+------------------+------+-----+---------------------+-----------------------------+
5 rows in set (0.00 sec)

mysql> select * from users;
+----+------+------+---------------------+---------------------+
| id | name | sex  | ctime               | mtime               |
+----+------+------+---------------------+---------------------+
|  1 | neo  | Y    | 0000-00-00 00:00:00 | 2013-05-17 18:05:09 |
|  2 | zen  | Y    | 0000-00-00 00:00:00 | 2013-05-17 18:05:11 |
|  3 | lily | N    | 0000-00-00 00:00:00 | 2013-05-17 18:05:22 |
+----+------+------+---------------------+---------------------+
3 rows in set (0.01 sec)

FEDERATED 与 mysqldump 问题！

切记，mysqldump 只会dump出使用FEDERATED引擎表的结构,不会包含数据。

12.9.2. BLACKHOLE

CREATE TABLE test(id INT, val CHAR(10)) ENGINE = BLACKHOLE;


第 13 章 DML (Data Manipulation Language)

目录

13.1. INSERT
13.1.1. INSERT INTO ... SELECT
SELECT - retrieve data from the a database
INSERT - insert data into a table
UPDATE - updates existing data within a table
DELETE - deletes all records from a table, the space for the records remain
CALL - call a PL/SQL or Java subprogram
EXPLAIN PLAN - explain access path to data
LOCK TABLE - control concurrency

13.1. INSERT

13.1.1. INSERT INTO ... SELECT

SET @OLDTMP_SQL_MODE=@@SQL_MODE, SQL_MODE='';
DELIMITER //
CREATE TRIGGER `members_mobile_insert` BEFORE INSERT ON `members_mobile` FOR EACH ROW BEGIN
    insert into members_location(id,province,city) select NEW.id,mobile_location.province,mobile_location.city from  mobile_location where mobile_location.id = md5(LEFT(NEW.number, 7));
END//
DELIMITER ;
SET SQL_MODE=@OLDTMP_SQL_MODE;


第 14 章 SQL Statement Syntax

Structured Query Language

目录

14.1. DISTINCT
14.2. group by
14.3. replace
14.4. OUTFILE/LOAD DATA INFILE
14.4.1. Export data to CSV from MySQL
14.5. SQL 92
14.6. Functions and Operators
14.6.1. COUNT
14.6.2. UUID()
14.6.3. String
14.6.3.1. LEFT/RIGHT
14.6.3.2. RPAD/LPAD
14.6.3.3. CONCAT
14.6.3.4. GROUP_CONCAT
14.6.3.5. replace
14.6.4. Date and Time
14.6.4.1. year/month/day hour:minite:second
14.6.4.2. Unix time
14.6.4.3. DATE_FORMAT
14.6.4.4. DATE_SUB/DATE_ADD
14.6.4.5. datediff / timediff
14.7. 事务处理和锁定语句
14.7.1. 排他锁
14.7.2. 表的加锁与解锁
14.7.3. 事务所用到的表
14.8. MySQL 专有命令
14.8.1. SQL_NO_CACHE

14.1. DISTINCT

SELECT DISTINCT user.name FROM user

SELECT DISTINCT user.name FROM user


14.2. group by

统计重复的手机号吗

select * from (select count(mobile) as c, mobile from member where length(mobile) >= 11 group by mobile) as m where m.c > 1;


14.3. replace

replace into

replace into (id) value('1')


14.4. OUTFILE/LOAD DATA INFILE

查询结果输出到文件

SELECT * FROM tablename INTO OUTFILE '/tmp/tablename.txt';

使用tee将屏幕输出到文件

mysql>tee /home/neo/screen.txt
mysql>select * from user;
mysql>exit

SELECT * INTO OUTFILE '/home/mark/Orders.txt'
    FIELDS
    TERMINATED BY = ','
    FROM Orders
    WHERE Order_Date >= '2000-01-01'

LOAD DATA INFILE 'data.txt' INTO TABLE db2.my_table;

14.4.1. Export data to CSV from MySQL

SELECT *
INTO OUTFILE '/tmp/products.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM products


14.5. SQL 92

insert + select

insert into product_type_commission select id,5,1,1,0,0,0,0,0,0 from product_type where title='notebook' and is_physical=0;

update table1,table2

begin;
ALTER TABLE `customer` ADD COLUMN `cutoff_time` TIMESTAMP NOT NULL default '0000-00-00 00:00:00';
update customer,agent set customer.cutoff_time = agent.cutoff_time where customer.id = agent.id;
ALTER TABLE `agent` DROP COLUMN `cutoff_time`;
commit;

update table1 set field1 = (select value from table2)

UPDATE
    transaction
SET
    transaction.total_sold_price = (
        SELECT
            SUM(transaction_item.price)
    FROM
            transaction_item
        WHERE transaction_item.transaction_id = 100
    )
WHERE
    transaction.id = 100

update table1, (select * from other) as table2 set table1.field1 = table2.field1

UPDATE
    transaction,(	SELECT
                        SUM(product_item.bought_price) AS total_bought_price, transaction_item.transaction_id
                    FROM
                        transaction_item
                    WHERE
                        transaction_item.transaction_id IN ( '123','456' )
                 ) as total
SET
    transaction.total_bought_price = total.total_bought_price
WHERE
    transaction.id = total.transaction_id

join + subquery

select u.*,t.category,t.items,t.[property] from tb_sysregchkusers as u left join (select a.items as category, b.* from (select id, items from tb_sysregchktask where categoryid=0) as a left join tb_sysregchktask as b on b.categoryid=a.id ) as t on u.taskID=t.id

select * from tb_sysregchklog where CONVERT(datetime,CONVERT(varchar(10),checkTime,120)) between convert(datetime,'2007-12-12') and convert(datetime,'2007-12-12')

select DISTINCT user_point_history.user_id,user.username,
(select count(id) from transaction where id = user_point_history.transaction_id) as transactions,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type != 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_earned,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type = 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_redeemed
from user_point_history,user where user_point_history.user_id = user.id;

(total_points_earned - total_points_redeemed) as current_balance_points

select user_id, username, transactions, total_points_earned, total_points_redeemed, (total_points_earned - total_points_redeemed) as current_balance_points
from (select DISTINCT user_point_history.user_id,user.username,
(select count(id) from transaction where id = user_point_history.transaction_id) as transactions,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type != 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_earned,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type = 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_redeemed
from user_point_history,user where user_point_history.user_id = user.id) as user_performance;

subquery作为一个字段使用

select product_type_attribute.*,(select 'selected' from product_type_attribute_set where product_type_attribute_set.product_type_attribute_id = product_type_attribute.id and product_type_attribute_set.product_type_id = 26) as selected
from product_type_attribute;


14.6. Functions and Operators

14.6.1. COUNT

count()

SELECT (SELECT count(1) FROM ecs_category) as 'Export category count',
    (SELECT count(1) FROM ecs_goods) as 'Goods count',
    (SELECT count(1) FROM ecs_goods_attr) as 'Attr count';

14.6.2. UUID()

SELECT UUID(),LENGTH(UUID()),UUID_SHORT(), LENGTH(UUID_SHORT());

14.6.3. String

14.6.3.1. LEFT/RIGHT

LEFT(str,len)

mysql> select left(concat('1','0000000'),5) as number;
+--------+
| number |
+--------+
| 10000  |
+--------+
1 row in set (0.00 sec)

RIGHT(str,len)

mysql> select right(concat('0000000','1'),5) as number;
+--------+
| number |
+--------+
| 00001  |
+--------+
1 row in set (0.00 sec)

14.6.3.2. RPAD/LPAD

补齐长度用'0'填充

RPAD(str,len,padstr)
mysql> select rpad('10',5,'0') as txt;
+-------+
| txt   |
+-------+
| 10000 |
+-------+
1 row in set (0.01 sec)
LPAD(str,len,padstr)
mysql> select lpad('10',5,'0') as txt;
+-------+
| txt   |
+-------+
| 00010 |
+-------+
1 row in set (0.00 sec)

14.6.3.3. CONCAT

CONCAT(str1,str2,...)

mysql> select concat('Neo',' ','Chen') as Name;
+----------+
| Name     |
+----------+
| Neo Chen |
+----------+
1 row in set (0.00 sec)

14.6.3.4. GROUP_CONCAT

mysql> select GROUP_CONCAT(CONVERT( username , CHAR (16)) order by username desc) as username from test;
+-------------------------------------------+
| username                                  |
+-------------------------------------------+
| jam,jam2,john,john2,john3,neo,neo1,neo2   |
+-------------------------------------------+
6 rows in set, 1 warning (0.01 sec)

14.6.3.5. replace

select replace(goods_desc,':8000','') from ecs_goods;

update ecs_goods set goods_desc=replace(goods_desc,':8000','');

14.6.4. Date and Time

14.6.4.1. year/month/day hour:minite:second

mysql> select year('2012-03-20');
+--------------------+
| year('2012-03-20') |
+--------------------+
|               2012 |
+--------------------+
1 row in set (0.00 sec)

mysql> select month('2012-03-20');
+---------------------+
| month('2012-03-20') |
+---------------------+
|                   3 |
+---------------------+
1 row in set (0.00 sec)

mysql> select day('2012-03-20');
+-------------------+
| day('2012-03-20') |
+-------------------+
|                20 |
+-------------------+
1 row in set (0.00 sec)

mysql> select hour('12:30:55');
+------------------+
| hour('12:30:55') |
+------------------+
|               12 |
+------------------+
1 row in set (0.00 sec)

mysql> select minute('12:30:55');
+--------------------+
| minute('12:30:55') |
+--------------------+
|                 30 |
+--------------------+
1 row in set (0.00 sec)

mysql> select second('12:30:55');
+--------------------+
| second('12:30:55') |
+--------------------+
|                 55 |
+--------------------+
1 row in set (0.00 sec)

14.6.4.2. Unix time

mysql> SELECT UNIX_TIMESTAMP('2005-03-27 02:00:00');
+---------------------------------------+
| UNIX_TIMESTAMP('2005-03-27 02:00:00') |
+---------------------------------------+
|                            1111885200 |
+---------------------------------------+
mysql> SELECT FROM_UNIXTIME(1111885200);
+---------------------------+
| FROM_UNIXTIME(1111885200) |
+---------------------------+
| 2005-03-27 03:00:00       |
+---------------------------+

SELECT UNIX_TIMESTAMP('2012-01-01 00:00:00');
SELECT UNIX_TIMESTAMP('2012-07-30 00:00:00');
SELECT UNIX_TIMESTAMP();
SELECT UNIX_TIMESTAMP('2009-08-06') ;
SELECT UNIX_TIMESTAMP( curdate( ) );

select FROM_UNIXTIME(UNIX_TIMESTAMP('2012-07-30 00:00:00'), '%Y-%m-%d');
SELECT FROM_UNIXTIME( 1249488000, '%Y年%m月%d日' );

select FROM_UNIXTIME(createtime, '%m') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2012-01-01 00:00:00') and UNIX_TIMESTAMP('2012-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m');
select FROM_UNIXTIME(createtime, '%m') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2011-01-01 00:00:00') and UNIX_TIMESTAMP('2011-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m');

select FROM_UNIXTIME(createtime, '%m-%d') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2011-01-01 00:00:00') and UNIX_TIMESTAMP('2011-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m-%d');
select FROM_UNIXTIME(createtime, '%m-%d') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2012-01-01 00:00:00') and UNIX_TIMESTAMP('2012-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m-%d');

14.6.4.3. DATE_FORMAT

DATE_FORMAT() 函数用于以不同的格式显示日期/时间数据。

语法
DATE_FORMAT(date,format)
date 参数是合法的日期。format 规定日期/时间的输出格式。

可以使用的格式有：
格式	描述
%a	缩写星期名
%b	缩写月名
%c	月，数值
%D	带有英文前缀的月中的天
%d	月的天，数值(00-31)
%e	月的天，数值(0-31)
%f	微秒
%H	小时 (00-23)
%h	小时 (01-12)
%I	小时 (01-12)
%i	分钟，数值(00-59)
%j	年的天 (001-366)
%k	小时 (0-23)
%l	小时 (1-12)
%M	月名
%m	月，数值(00-12)
%p	AM 或 PM
%r	时间，12-小时（hh:mm:ss AM 或 PM）
%S	秒(00-59)
%s	秒(00-59)
%T	时间, 24-小时 (hh:mm:ss)
%U	周 (00-53) 星期日是一周的第一天
%u	周 (00-53) 星期一是一周的第一天
%V	周 (01-53) 星期日是一周的第一天，与 %X 使用
%v	周 (01-53) 星期一是一周的第一天，与 %x 使用
%W	星期名
%w	周的天 （0=星期日, 6=星期六）
%X	年，其中的星期日是周的第一天，4 位，与 %V 使用
%x	年，其中的星期一是周的第一天，4 位，与 %v 使用
%Y	年，4 位
%y	年，2 位

实例

下面的脚本使用 DATE_FORMAT() 函数来显示不同的格式。我们使用 NOW() 来获得当前的日期/时间：
DATE_FORMAT(NOW(),'%b %d %Y %h:%i %p')
DATE_FORMAT(NOW(),'%m-%d-%Y')
DATE_FORMAT(NOW(),'%d %b %y')
DATE_FORMAT(NOW(),'%d %b %Y %T:%f')

SELECT DATE_FORMAT(NOW(),'%Y-%m-%d');

select DATE_FORMAT(asctime,'%Y-%m-%d') as Date, count(1) as Count from logging where tag='www' and facility='login' group by DATE_FORMAT(asctime,'%Y-%m-%d') order by asctime desc;

14.6.4.4. DATE_SUB/DATE_ADD

当前时间向后推10天

mysql> select DATE_SUB(now(), INTERVAL 240 HOUR);
+------------------------------------+
| DATE_SUB(now(), INTERVAL 240 HOUR) |
+------------------------------------+
| 2012-03-09 10:26:03                |
+------------------------------------+
1 row in set (0.00 sec)

mysql> select DATE_SUB(now(), INTERVAL 24 HOUR);
+-----------------------------------+
| DATE_SUB(now(), INTERVAL 24 HOUR) |
+-----------------------------------+
| 2012-03-18 10:28:43               |
+-----------------------------------+
1 row in set (0.00 sec)

DELETE from Message where created < DATE_sub(now(), INTERVAL 240 HOUR);

select * from PRICES_HISTORY where DATE_FORMAT(TIME ,GET_FORMAT(DATE,'ISO')) = (
select if ( WEEKDAY(CURRENT_DATE())=6 , DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) , CURRENT_DATE())
)
14.6.4.4.1. DATE_ADD
SELECT DATE_ADD('1998-01-02', INTERVAL 31 DAY);

14.6.4.5. datediff / timediff

计算时间差，两个时间相减结果

mysql> select timediff('22:20:00','17:30:00');
+---------------------------------+
| timediff('22:20:00','17:30:00') |
+---------------------------------+
| 04:50:00                        |
+---------------------------------+
1 row in set (0.00 sec)

mysql> select datediff('2008-08-08 12:00:00', '2008-08-01 00:00:00');
+--------------------------------------------------------+
| datediff('2008-08-08 12:00:00', '2008-08-01 00:00:00') |
+--------------------------------------------------------+
|                                                      7 |
+--------------------------------------------------------+
1 row in set (0.00 sec)


14.7. 事务处理和锁定语句

Transactional and Locking Statements

开始事务 begin、start transaction 或者 set autocommit=0

事务的特征：原子性（Atomiocity）、一致性（Consistency）、隔离性（Isolation）和持久性（Durability），这四个特性简称ACID特性。
    原子性：事务是数据库的逻辑工作单位，事务中包括的所有操作要么都做，要么都不做。
    一致性：事务执行的结果必须是使数据库从一个一致性的状态变到另外一个一致性状态。
    隔离性：一个事务的执行不能被其他事务干扰。即一个事务内部的操作及使用的数据对其他事务是隔离的，并发执行的各个事务之间互相不干扰。
    持久性：一个事务一旦成功提交，对数据库中数据的修改就是持久性的。接下来其他的其他操作或故障不应该对其执行结果有任何影响。

事务隔离模式

     1) READ UNCOMMITED
    SELECT的时候允许脏读，即SELECT会读取其他事务修改而还没有提交的数据。

    2)READ COMMITED
    SELECT的时候无法重复读，即同一个事务中两次执行同样的查询语句，若在第一次与第二次查询之间时间段，其他事务又刚好修改了其查询的数据且提交了，则两次读到的数据不一致。

    3) REPEATABLE READ
    SELECT的时候可以重复读，即同一个事务中两次执行同样的查询语句，得到的数据始终都是一致的。实现的原理是，在一个事务对数据行执行读取或写入操作时锁定了这些数据行。
    但是这种方式又引发了幻想读的问题。因为只能锁定读取或写入的行，不能阻止另一个事务插入数据，后期执行同样的查询会产生更多的结果。

    4)SERIALIZABLE
    与可重复读的唯一区别是，默认把普通的SELECT语句改成SELECT …. LOCK IN SHARE MODE。即为查询语句涉及到的数据加上共享琐，阻塞其他事务修改真实数据。
    serializable模式中，事务被强制为依次执行。这是SQL标准建议的默认行为。

  查看InnoDB系统级别的事务隔离级别：mysql> SELECT @@global.tx_isolation;
  查看InnoDB会话级别的事务隔离级别：mysql> SELECT @@tx_isolation;
  修改InnoDB系统级别的事务隔离级别：mysql> set global transaction isolation level read committed;
  修改InnoDB会话级别的事务隔离级别：mysql> set session transaction isolation level read committed;

锁机制

    1) 共享锁：由读表操作加上的锁，加锁后其他用户只能获取该表或行的共享锁，不能获取排它锁，也就是说只能读不能写
    2) 排它锁：由写表操作加上的锁，加锁后其他用户不能获取该表或行的任何锁，典型是mysql事务中的

    锁的范围:
    行锁: 对某行记录加上锁
    表锁: 对整个表加上锁

共享锁(share mode), 排他锁(for update)

14.7.1. 排他锁

下面做作一个实验，验证锁的效果

终端一,首先进入事务状态然后运行下面语句

mysql> begin;
Query OK, 0 rows affected (0.00 sec)

mysql> select * from t1 where id='3' for update;
+----+--------+---------------------+---------------------+
| id | name   | ctime               | mtime               |
+----+--------+---------------------+---------------------+
|  3 | test   | 0000-00-00 00:00:00 | 2013-01-14 13:05:41 |
+----+--------+---------------------+---------------------+
1 row in set (0.00 sec)

终端二, 查询表中数据

mysql> select * from t1;
+----+--------+---------------------+---------------------+
| id | name   | ctime               | mtime               |
+----+--------+---------------------+---------------------+
|  1 | neo    | 0000-00-00 00:00:00 | 2013-01-14 13:00:00 |
|  2 | zen    | 0000-00-00 00:00:00 | 2013-01-14 13:00:43 |
|  3 | test   | 0000-00-00 00:00:00 | 2013-01-14 13:05:41 |
+----+--------+---------------------+---------------------+
3 rows in set (0.00 sec)

增加“for update”查询非锁定记录

mysql> select * from t1 where id=2 for update;
+----+------+---------------------+---------------------+
| id | name | ctime               | mtime               |
+----+------+---------------------+---------------------+
|  2 | zen  | 0000-00-00 00:00:00 | 2013-01-14 13:00:43 |
+----+------+---------------------+---------------------+
1 row in set (0.00 sec)

查询被锁定记录

mysql> select * from t1 where id=3 for update;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

查询所有记录，因为记录中包含了id=3那条，所以也不允许查询。

mysql> select * from t1 for update;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

测试修改记录

mysql> UPDATE `t1` SET `name`='testaa' WHERE  `id`=3;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

提示

在没有出现ERROR 1205 (HY000)的这段时间，只要终端一中执行commit,rollback锁就释放了.终端二中的语句就会运行。

select trx_query from information_schema.innodb_trx; 可以查看被锁的SQL语句

14.7.2. 表的加锁与解锁

LOCK TABLES tablename WRITE;
LOCK TABLES tablename READ;

...
...

UNLOCK TABLES;

14.7.3. 事务所用到的表

information_schema

select * from innodb_trx;
select * from innodb_lock_waits;
select * from innodb_locks;


14.8. MySQL 专有命令

14.8.1. SQL_NO_CACHE

SELECT /*!40001 SQL_NO_CACHE */ * FROM table


第 15 章 Optimization

目录

15.1. Buffering and Caching
15.1.1. Query Cache SELECT Options
15.2. where 优化

15.1. Buffering and Caching

查看缓存是否开启

MySQL> select @@query_cache_type;
MySQL> show variables like 'query_cache_type';

开启与关闭缓存

MySQL> set query_cache_type=on;
MySQL> set query_cache_type=off;

查看缓存状态

show variables like 'have_query_cache';

查询缓存的大小

MySQL> select @@global.query_cache_size;
MySQL> select @@query_cache_size;

查看最大缓存限制，如果集大于该数则不缓存。

MySQL> select @@global.query_cache_limit;

清除缓存/重置缓存

MySQL> flush tables;
MySQL> flush query cache;
MySQL> reset query cache;

查询缓存性能

MySQL> show status like 'qcache%';

MySQL> show status like 'qcache_q%';
+-------------------------+-------+
| Variable_name | Value |
+-------------------------+-------+
| Qcache_queries_in_cache | 1 |
+-------------------------+-------+
1 row in set (0.00 sec)

MySQL> show status like 'qcache_f%';
+--------------------+----------+
| Variable_name | Value |
+--------------------+----------+
| Qcache_free_blocks | 1 |
| Qcache_free_memory | 16766728 |
+--------------------+----------+
2 rows in set (0.00 sec)

15.1.1. Query Cache SELECT Options

Two query cache-related options may be specified in SELECT statements:

SQL_CACHE

The query result is cached if it is cacheable and the value of the query_cache_type system variable is ON or DEMAND.

SQL_NO_CACHE

The query result is not cached.

Examples:

SELECT SQL_CACHE id, name FROM customer;
SELECT SQL_NO_CACHE id, name FROM customer;

SELECT /*! SQL_NO_CACHE */ stuff FROM table

例 15.1. SQL_CACHE 测试

下面的例子中你将看到缓存变化

flush tables;
show status like 'qcache_q%';
select sql_cache * from members limit 5;
show status like 'qcache_q%';
select sql_cache * from members limit 10;
show status like 'qcache_q%';

显示当前缓存中的信息数量：

MySQL> show status like 'qcache_q%';


15.2. where 优化

where 条件的顺序影响查询速度

EXPLAIN select *,from_unixtime(sendtime) from sms where id='461539' and content like '13%';
/* 0 rows affected, 1 rows found. Duration for 1 query: 0.218 sec. */

EXPLAIN select *,from_unixtime(sendtime) from sms where content like '13%' and id='461539';
/* 0 rows affected, 1 rows found. Duration for 1 query: 0.717 sec. */


第 16 章 MySQL Connectors

目录

16.1. JDBC
16.2. ODBC
16.3. MySQL native driver for PHP - mysqlnd
16.4. python-mysqldb

16.1. JDBC

JDBC connection settings

jdbc:mysql://hostname:port/database?autoReconnect=true&useUnicode=true&characterEncoding=utf8

confluence.cfg.xml

<property name="hibernate.connection.url">jdbc:mysql://hostname:port/database?autoReconnect=true&amp;useUnicode=true&amp;characterEncoding=utf8</property>


16.2. ODBC


16.3. MySQL native driver for PHP - mysqlnd


16.4. python-mysqldb

$ apt-cache search python | grep mysql
python-mysqldb - A Python interface to MySQL
python-mysqldb-dbg - A Python interface to MySQL (debug extension)

$ sudo apt-get install python-mysqldb

# -*- coding: utf-8 -*-
#mysqldb
import time, MySQLdb

#连接
conn=MySQLdb.connect(host="localhost",user="root",passwd="",db="test",charset="utf8")
cursor = conn.cursor()

#写入
sql = "insert into user(name,created) values(%s,%s)"
param = ("neo",int(time.time()))
n = cursor.execute(sql,param)
print n

#更新
sql = "update user set name=%s where id=3"
param = ("jam")
n = cursor.execute(sql,param)
print n

#查询
n = cursor.execute("select * from user")
for row in cursor.fetchall():
    for r in row:
        print r

#删除
sql = "delete from user where name=%s"
param =("neo")
n = cursor.execute(sql,param)
print n
cursor.close()

#关闭
conn.close()


第 17 章 MySQL GUI/Web Manager

目录

17.1. HeidiSQL
17.2. Toad for MySQL Freeware
17.3. phpMyAdmin - MySQL web administration tool
17.4. Maatkit Essential command-line utilities for MySQL

17.1. HeidiSQL

http://www.heidisql.com/


17.2. Toad for MySQL Freeware

http://toadsoft.veriomigrations.com/


17.3. phpMyAdmin - MySQL web administration tool

homepage: http://www.phpmyadmin.net/

$ wget http://nchc.dl.sourceforge.net/sourceforge/phpmyadmin/phpMyAdmin-3.1.3.1-all-languages.tar.bz2
$ tar jxvf phpMyAdmin-3.1.3.1-all-languages.tar.bz2
$ ln -s phpMyAdmin-3.1.3.1-all-languages phpMyAdmin


17.4. Maatkit Essential command-line utilities for MySQL

http://www.maatkit.org/


第 18 章 Miscellaneous

目录

18.1. Multi-Master Replication Manager for MySQL
18.2. HandlerSocket
18.3. MariaDB
18.4. Maatkit

18.1. Multi-Master Replication Manager for MySQL


18.2. HandlerSocket


18.3. MariaDB

http://mariadb.org/


18.4. Maatkit

http://www.maatkit.org/


第 19 章 FAQ

目录

19.1. 数据库内容替换
19.2. 查看错误代码
19.2.1. ERROR 1153 (08S01) at line 3168: Got a packet bigger than 'max_allowed_packet' bytes
19.2.2. ERROR 1129 (00000): Host 'XXXXXX' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
19.3. 临时表是否需要建索引
19.4. Kill 脚本

19.1. 数据库内容替换

#!/bin/bash
HOST='localhost'
USER='neo'
PASS='chen'

SDB='neo'
TDB='netkiller'
MYSQLDUMP="mysqldump"
MYSQLDUMPOPTS="-h${HOST} -u${USER} -p${PASS}"

MYSQL="mysql"
MYSQLOPTS="-h${HOST} -u${USER} -p${PASS}"
#SED="sed -e 's/netkiller\.8800\.org/netkiller\.sf\.net/g' -e 's/陈景峰/景峰/g' -e 's/Neo/Netkiller/g'"

$MYSQL $MYSQLOPTS <<SQL
DROP DATABASE $TDB;
CREATE DATABASE $TDB DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
SQL

$MYSQLDUMP $MYSQLDUMPOPTS ${SDB} | sed -e 's/netkiller\.8800\.org/netkiller\.sf\.net/g' -e 's/陈景峰/景峰/g' -e 's/Neo/Netkiller/g' | $MYSQL $MYSQLOPTS ${TDB}
#echo "$MYSQLDUMP $MYSQLDUMPOPTS ${SDB} | $SED | $MYSQL $MYSQLOPTS ${TDB}"


19.2. 查看错误代码

mysql> \! perror 6
OS error code   6:  No such device or address

19.2.1. ERROR 1153 (08S01) at line 3168: Got a packet bigger than 'max_allowed_packet' bytes

max_allowed_packet=500M

19.2.2. ERROR 1129 (00000): Host 'XXXXXX' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'

连接在中途被中断了的连接请求。在 max_connect_errors次失败请求后，mysql阻止该主机进一步的连接，直到管理员执行命令 mysqladmin flush-hosts。

mysql> flush hosts;

set global max_connect_errors=1000;

max_connect_errors=10000


19.3. 临时表是否需要建索引

答案：要


19.4. Kill 脚本

查询出锁定的表

SELECT concat('KILL ',id,';') FROM information_schema.processlist WHERE user='root';

SELECT concat('KILL ',id,';') FROM information_schema.processlist WHERE command='Locked' and user='root';

SELECT concat('KILL ',id,';') FROM information_schema.processlist WHERE command='Locked' and user='root' and db='test';

拼装kill命令后输入到kill.sql, source 将从kill.sql读取sql命令并执行。

SELECT concat('KILL ',id,';') FROM information_schema.processlist WHERE user='root' INTO OUTFILE '/tmp/kill.sql';

source /tmp/kill.sql;

mysqladmin -uroot -p processlist | grep Sleep |awk '{if (length($2) > 1) print "Kill "$2}'|xargs mysqladmin -uroot kill


部分 III. PostgreSQL

目录

20. PostgreSQL 安装
20.1. Ubuntu 12.04.1 LTS 安装
20.2. YUM 默认源安装
20.3. PostgreSQL YUM 源安装
20.4. PostgreSQL 配置
20.4.1. postgresql.conf
20.4.2. pg_hba.conf
20.5. 创建dba用户
21. Administration
21.1. User 用户管理
21.1.1. SUPERUSER
21.1.2. 删除用户
21.2. Database
21.2.1. 删除数据库
21.3. Table
21.4. Backup / Restore
21.4.1. Backup
21.4.2. Restore
21.4.3. import
21.4.4. PostgreSQL for windows
21.4.5. 大型数据库备份
22. PostgreSQL 系统表
22.1. pg_available_extensions
22.2. pg_shadow
23. DDL - Data Definition Language
23.1. Database
23.1.1. create
23.1.2. drop
23.1.3. Alter
23.1.4. Rename
23.1.5. CHARACTER
23.1.6. show create database
23.2. Table
23.2.1. create table ... select
23.2.2. modifiy table
23.2.3. TEMPORARY Table
23.2.4. Collate
23.2.5. CHARACTER
23.2.6. DEFAULT
23.2.7. KEY
23.2.7.1. PRIMARY KEY
23.2.8. COMMENT
23.3. Index
23.3.1. SHOW INDEX
23.3.2. CREATE INDEX
23.3.3. DROP INDEX
23.3.4. rebuild
23.4. 外键(Foreign Key)
23.4.1. FOREIGN KEY (RESTRICT)
23.5. View
23.6. Trigger
23.6.1. create trigger
23.6.2. drop trigger
23.6.3. show triggers
23.6.4. EXAMPLE
23.7. EVENT
23.7.1. 启用 EVENT
23.7.2. 创建 EVENT
23.7.3. 禁用/启用
23.7.4. show events
23.8. Partitioning
23.8.1. RANGE
23.8.2. LIST
23.8.3. HASH
23.8.3.1. LINEAR HASH
23.8.4. KEY分区
23.8.5. 分区管理
23.8.5.1. 新增分区
23.8.5.2. 删除分区
23.8.5.3. 重建分区
23.8.5.4. 分区维护
23.8.6. EXPLAIN PARTITIONS
23.8.7. SHOW CREATE TABLE
23.8.8. INFORMATION_SCHEMA.partitions 表
23.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
23.9. Engine
23.9.1. FEDERATED
23.9.2. BLACKHOLE
24. DML (Data Manipulation Language)
24.1. INSERT
24.1.1. INSERT INTO ... SELECT
25. SQL Statement Syntax
25.1. DISTINCT
25.2. group by
25.3. replace
25.4. OUTFILE/LOAD DATA INFILE
25.4.1. Export data to CSV from MySQL
25.5. SQL 92
25.6. Functions and Operators
25.6.1. COUNT
25.6.2. UUID()
25.6.3. String
25.6.3.1. LEFT/RIGHT
25.6.3.2. RPAD/LPAD
25.6.3.3. CONCAT
25.6.3.4. GROUP_CONCAT
25.6.3.5. replace
25.6.4. Date and Time
25.6.4.1. year/month/day hour:minite:second
25.6.4.2. Unix time
25.6.4.3. DATE_FORMAT
25.6.4.4. DATE_SUB/DATE_ADD
25.6.4.5. datediff / timediff
25.7. 事务处理和锁定语句
25.7.1. 排他锁
25.7.2. 表的加锁与解锁
25.7.3. 事务所用到的表
25.8. MySQL 专有命令
25.8.1. SQL_NO_CACHE
26. PostgreSQL GUI
26.1. pgAdmin III
26.2. phpPgAdmin
26.3. Monitoring
26.3.1. PgBadger
26.4. pgModeler - PostgreSQL Database Modeler
27. Barman Backup & recovery for PostgreSQL
28. pgbouncer - lightweight connection pooler for PostgreSQL
28.1. 安装 pgbouncer
28.1.1. Ubuntu
28.1.2. CentOS
28.2. 配置 pgbouncer
29. Foreign data wrappers
30. FAQ

PostgreSQL 实用实例参考


第 20 章 PostgreSQL 安装

目录

20.1. Ubuntu 12.04.1 LTS 安装
20.2. YUM 默认源安装
20.3. PostgreSQL YUM 源安装
20.4. PostgreSQL 配置
20.4.1. postgresql.conf
20.4.2. pg_hba.conf
20.5. 创建dba用户

20.1. Ubuntu 12.04.1 LTS 安装

安装环境

PostgreSQL 9.1

$ sudo apt-get install postgresql
$ sudo apt-get install postgresql

更改postgres管理员用户密码

$ sudo passwd postgres
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

然后切换到postgres用户环境

$ su - postgres
Password:
Added user postgres.

进入psql客户款, PostgreSQL的psql命令相当于sqlplus,mysql命令

$ psql
psql (9.1.6)
Type "help" for help.

postgres=#

退出\q

postgres=# \q


20.2. YUM 默认源安装

# yum search postgresql | grep ^postgresql
postgresql-odbc.x86_64 : PostgreSQL ODBC driver
postgresql.i686 : PostgreSQL client programs
postgresql.x86_64 : PostgreSQL client programs
postgresql-contrib.x86_64 : Contributed modules distributed with PostgreSQL
postgresql-devel.i686 : PostgreSQL development header files and libraries
postgresql-devel.x86_64 : PostgreSQL development header files and libraries
postgresql-docs.x86_64 : Extra documentation for PostgreSQL
postgresql-ip4r.x86_64 : IPv4 and IPv4 range index types for PostgreSQL
postgresql-jdbc.noarch : JDBC driver for PostgreSQL
postgresql-libs.i686 : The shared libraries required for any PostgreSQL clients
postgresql-libs.x86_64 : The shared libraries required for any PostgreSQL
postgresql-pgpool-II.i686 : Pgpool is a connection pooling/replication server
postgresql-pgpool-II.x86_64 : Pgpool is a connection pooling/replication server
postgresql-pgpool-II-recovery.x86_64 : PGPool recovery add-on for PostgreSQL
postgresql-plparrot.x86_64 : A PostgreSQL procedural language for the Parrot
postgresql-plperl.x86_64 : The Perl procedural language for PostgreSQL
postgresql-plpython.x86_64 : The Python procedural language for PostgreSQL
postgresql-plruby.x86_64 : PostgreSQL Ruby Procedural Language
postgresql-pltcl.x86_64 : The Tcl procedural language for PostgreSQL
postgresql-server.x86_64 : The programs needed to create and run a PostgreSQL
postgresql-test.x86_64 : The test suite distributed with PostgreSQL
postgresql_autodoc.noarch : PostgreSQL AutoDoc Utility
postgresql-pgpool-II-devel.i686 : The development files for pgpool-II
postgresql-pgpool-II-devel.x86_64 : The development files for pgpool-II
postgresql-plruby-doc.x86_64 : Documentation for plruby

默认YUM源上面只有 8.4.13

# yum info postgresql-server
Loaded plugins: fastestmirror, presto, refresh-packagekit
Loading mirror speeds from cached hostfile
 * base: centos.uhost.hk
 * epel: mirror01.idc.hinet.net
 * extras: centos.uhost.hk
 * updates: centos.uhost.hk
Installed Packages
Name        : postgresql-server
Arch        : x86_64
Version     : 8.4.13
Release     : 1.el6_3
Size        : 14 M
Repo        : installed
From repo   : base
Summary     : The programs needed to create and run a PostgreSQL server
URL         : http://www.postgresql.org/
License     : PostgreSQL
Description : The postgresql-server package includes the programs needed to create
            : and run a PostgreSQL server, which will in turn allow you to create
            : and maintain PostgreSQL databases.  PostgreSQL is an advanced
            : Object-Relational database management system (DBMS) that supports
            : almost all SQL constructs (including transactions, subselects and
            : user-defined types and functions). You should install
            : postgresql-server if you want to create and maintain your own
            : PostgreSQL databases and/or your own PostgreSQL server. You also need
            : to install the postgresql package.

# yum install postgresql-server

# chkconfig postgresql on
# service postgresql initdb
# service postgresql start


20.3. PostgreSQL YUM 源安装

CentOS 6.4 环境 YUM 地址 http://yum.postgresql.org/

# yum install http://yum.postgresql.org/9.2/redhat/rhel-6-x86_64/pgdg-centos92-9.2-6.noarch.rpm
# yum list postgres*

# yum install postgresql92-server postgresql92

chkconfig postgresql-9.2 on
service postgresql-9.2 initdb
service postgresql-9.2 start

配置文件的位置

# ls /var/lib/pgsql/9.2/data/*.conf
/var/lib/pgsql/9.2/data/pg_hba.conf  /var/lib/pgsql/9.2/data/pg_ident.conf  /var/lib/pgsql/9.2/data/postgresql.conf


20.4. PostgreSQL 配置

su 到 postgres 用户

$ su - postgres
Password:
$ pwd
/var/lib/postgresql
$

备份配置文件，防止修改过程中损毁

cp /etc/postgresql/9.1/main/postgresql.conf /etc/postgresql/9.1/main/postgresql.conf.original
cp /etc/postgresql/9.1/main/pg_hba.conf /etc/postgresql/9.1/main/pg_hba.conf.original

20.4.1. postgresql.conf

启用tcp/ip连接，去掉下面注释,修改为你需要的IP地址，默认为localhost

listen_addresses = 'localhost'

如果有多个网络适配器可以指定 'ip' 或 '*' 任何接口上的IP地址都可能listen.

$ sudo vim /etc/postgresql/9.1/main/postgresql.conf

listen_addresses = '*'

20.4.2. pg_hba.conf

pg_hba.conf配置文件的权限需要注意以下，-rw-r----- 1 postgres postgres 4649 Dec 5 18:00 pg_hba.conf

$ ll /etc/postgresql/9.1/main/
total 52
drwxr-xr-x 2 postgres postgres  4096 Dec  6 09:40 ./
drwxr-xr-x 3 postgres postgres  4096 Dec  5 18:00 ../
-rw-r--r-- 1 postgres postgres   316 Dec  5 18:00 environment
-rw-r--r-- 1 postgres postgres   143 Dec  5 18:00 pg_ctl.conf
-rw-r----- 1 postgres postgres  4649 Dec  5 18:00 pg_hba.conf
-rw-r----- 1 postgres postgres  1636 Dec  5 18:00 pg_ident.conf
-rw-r--r-- 1 postgres postgres 19259 Dec  5 18:00 postgresql.conf
-rw-r--r-- 1 postgres postgres   378 Dec  5 18:00 start.conf

pg_hba.conf配置文件负责访问权限控制

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     peer
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5
TYPE

local 本地使用unix/socket 方式连接, host 使用tcp/ip socket 方式连接

DATABASE

数据库名.

USER

用户名.

ADDRESS

允许连接的IP地址，可以使用子网掩码.

METHOD

认真加密方式.

下面我们做一个简单测试，首先配置pg_hba。conf文件

$ sudo vi /etc/postgresql/9.1/main/pg_hba.conf
host    *			dba         0.0.0.0/0       md5
host    test		test        0.0.0.0/0       md5

运行创建数据,用户 的SQL语句

CREATE ROLE test LOGIN PASSWORD 'test' NOSUPERUSER NOINHERIT NOCREATEDB NOCREATEROLE;

CREATE DATABASE test WITH OWNER = test ENCODING = 'UTF8' TABLESPACE = pg_default;

进入psql

$ psql
psql (9.1.6)
Type "help" for help.

postgres=# CREATE ROLE test LOGIN PASSWORD 'test' NOSUPERUSER NOINHERIT NOCREATEDB NOCREATEROLE;
CREATE ROLE
postgres=# CREATE DATABASE test WITH OWNER = test ENCODING = 'UTF8' TABLESPACE = pg_default;
CREATE DATABASE
postgres=# \q

使用psql登录

$ psql -hlocalhost -Utest test
Password for user test:
psql (9.1.6)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.

test=> \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 test      | test     | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
(4 rows)

test=>


20.5. 创建dba用户

创建一个远程维护数据库dba用户，具有创建数据库与创建用户的权限

CREATE USER dba PASSWORD 'dba' CREATEDB CREATEUSER;

进入psql

$ psql
psql (9.1.6)
Type "help" for help.

postgres=# CREATE USER dba PASSWORD 'dba' CREATEDB CREATEUSER;
CREATE ROLE
postgres=# \q

使用psql登录

$ psql -hlocalhost -Udba postgres
Password for user dba:
psql (9.1.6)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.

postgres=#


第 21 章 Administration

目录

21.1. User 用户管理
21.1.1. SUPERUSER
21.1.2. 删除用户
21.2. Database
21.2.1. 删除数据库
21.3. Table
21.4. Backup / Restore
21.4.1. Backup
21.4.2. Restore
21.4.3. import
21.4.4. PostgreSQL for windows
21.4.5. 大型数据库备份

21.1. User 用户管理

新建用户

CREATE ROLE woodart LOGIN PASSWORD 'chen'
  NOINHERIT
   VALID UNTIL 'infinity';

21.1.1. SUPERUSER

CREATE ROLE dba LOGIN
  PASSWORD 'your password'
  SUPERUSER INHERIT CREATEDB CREATEROLE REPLICATION;

21.1.2. 删除用户

dropuser -h192.168.1.1 -p 5432 -i -e testuser
User "testuser" and any owned databases will be permanently deleted.
Are you sure? (y/n) y


21.2. Database

$ sudo -u postgres createuser -W dbuser
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) y
Shall the new role be allowed to create more new roles? (y/n) n
Password:

$ sudo -u postgres createdb testdb -O dbuser

新建数据

CREATE DATABASE woodart
  WITH ENCODING='UTF8'
       OWNER=woodart;

CREATE DATABASE mydb ENCODING 'UTF8' TEMPLATE template0 OWNER dbuser TABLESPACE myspace;

21.2.1. 删除数据库

dropdb mydb

DROP DATABASE mydb


21.3. Table


21.4. Backup / Restore

21.4.1. Backup

本地备份

pg_dump --compress=9 --file=/backup/mydb.sql.gz mydb

Backup

pg_dump -i -h localhost -p 5432 -U user -Fc -c -f "your_db.backup" your_db

21.4.2. Restore

Restore

pg_restore -i -h localhost -p 5432 -U user -d your_db -v -c -Fc "your_db.backup"

21.4.3. import

导入

\i file.backup

postgres=# \i /home/neo/woodart.backup

21.4.4. PostgreSQL for windows

local -> remote

set PG_HOME="C:\Program Files\PostgreSQL\8.1\bin"
%PG_HOME%\pg_dump.exe -i -h localhost -p 5432 -U woodart -Fc -c -f "woodart.backup" woodart
%PG_HOME%\pg_restore.exe -i -h woodart.8800.org -p 5432 -U woodart -d woodart -v -c -Fc "woodart.backup"

21.4.5. 大型数据库备份

split 命令可以将大型文件切成小块以适应文件系统限制的单个文件大小。

分割,每650M为一个文件，试用于光盘备份

$ pg_dump dbname | split -b 650m - filename

合并，并且恢复到数据库中。

$ createdb dbname
$ cat filename* | psql dbname


第 22 章 PostgreSQL 系统表

目录

22.1. pg_available_extensions
22.2. pg_shadow

22.1. pg_available_extensions

SELECT * FROM pg_available_extensions;

postgres=# SELECT * FROM pg_available_extensions;
  name   | default_version | installed_version |           comment
---------+-----------------+-------------------+------------------------------
 plpgsql | 1.0             | 1.0               | PL/pgSQL procedural language
(1 row)


22.2. pg_shadow

postgres=# SELECT * FROM pg_shadow;
 usename  | usesysid | usecreatedb | usesuper | usecatupd | userepl |               passwd                | valuntil | useconfig
----------+----------+-------------+----------+-----------+---------+-------------------------------------+----------+-----------
 postgres |       10 | t           | t        | t         | t       |                                     |          |
 wechat   |    16385 | f           | f        | f         | f       | md55bf335eef50073dbc3a4a2103e7b3757 |          |
 test     |    16387 | f           | f        | f         | f       | md505a671c66aefea124cc08b76ea6d30bb |          |
 dba      |    16384 | t           | t        | t         | f       | md5a1453e5a69943d1674eb2f7100efa04c |          |
(4 rows)


第 23 章 DDL - Data Definition Language

目录

23.1. Database
23.1.1. create
23.1.2. drop
23.1.3. Alter
23.1.4. Rename
23.1.5. CHARACTER
23.1.6. show create database
23.2. Table
23.2.1. create table ... select
23.2.2. modifiy table
23.2.3. TEMPORARY Table
23.2.4. Collate
23.2.5. CHARACTER
23.2.6. DEFAULT
23.2.7. KEY
23.2.7.1. PRIMARY KEY
23.2.8. COMMENT
23.3. Index
23.3.1. SHOW INDEX
23.3.2. CREATE INDEX
23.3.3. DROP INDEX
23.3.4. rebuild
23.4. 外键(Foreign Key)
23.4.1. FOREIGN KEY (RESTRICT)
23.5. View
23.6. Trigger
23.6.1. create trigger
23.6.2. drop trigger
23.6.3. show triggers
23.6.4. EXAMPLE
23.7. EVENT
23.7.1. 启用 EVENT
23.7.2. 创建 EVENT
23.7.3. 禁用/启用
23.7.4. show events
23.8. Partitioning
23.8.1. RANGE
23.8.2. LIST
23.8.3. HASH
23.8.3.1. LINEAR HASH
23.8.4. KEY分区
23.8.5. 分区管理
23.8.5.1. 新增分区
23.8.5.2. 删除分区
23.8.5.3. 重建分区
23.8.5.4. 分区维护
23.8.6. EXPLAIN PARTITIONS
23.8.7. SHOW CREATE TABLE
23.8.8. INFORMATION_SCHEMA.partitions 表
23.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
23.9. Engine
23.9.1. FEDERATED
23.9.2. BLACKHOLE

23.1. Database

23.1.1. create

Creating a UTF-8 database

CREATE DATABASE db_name DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

Create a UTF-8 database with binary UTF-8 collation.

CREATE DATABASE dbname CHARACTER SET utf8 COLLATE utf8_bin;

23.1.2. drop

DROP DATABASE db_name;

23.1.3. Alter

ALTER DATABASE dbname DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

23.1.4. Rename

RENAME {DATABASE | SCHEMA} db_name TO new_db_name;

before 5.0 version

[neo@development ~]$ mysqldump -uroot -pchen db_old | mysql -uroot -pchen db_new

23.1.5. CHARACTER

ALTER DATABASE <database_name> CHARACTER SET utf8;

23.1.6. show create database

mysql> show create database dbname;
+----------+-------------------------------------------------------------------+
| Database | Create Database                                                   |
+----------+-------------------------------------------------------------------+
| dbname   | CREATE DATABASE `dbname` /*!40100 DEFAULT CHARACTER SET utf8 */   |
+----------+-------------------------------------------------------------------+
1 row in set (0.00 sec)


23.2. Table

23.2.1. create table ... select

创建空表

create table admin_user_history select * from admin_user where 1 <> 1;

创建有数据的表

create table admin_user_history select * from admin_user;

23.2.2. modifiy table

modifiy table

ALTER TABLE ecs_users add user_picture varchar(255);

23.2.3. TEMPORARY Table

临时表将在你连接期间存在。一旦断开时将自动删除表并释放所用的空间。你在连接期间删除该表也同样释放空间。

CREATE TEMPORARY TABLE tmp_table (
    key VARCHAR(10) NOT NULL,
    value INTEGER NOT NULL
)

声明临时表是一个HEAP表，允许你指定在内存中创建它

CREATE TEMPORARY TABLE tmp_mem_table (
    key VARCHAR(10) NOT NULL,
    value INTEGER NOT NULL
) TYPE = HEAP

23.2.4. Collate

ALTER TABLE `tmp_cats`  COLLATE='utf8_general_ci',  CONVERT TO CHARSET utf8;

23.2.5. CHARACTER

ALTER TABLE <table_name> CONVERT TO CHARACTER SET utf8;

23.2.6. DEFAULT

更新时间

`mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更改时间',

CREATE TABLE `bank_account` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '自增唯一ID',
    `bank_name` VARCHAR(255) NOT NULL DEFAULT '0' COMMENT '银行名字(Bank Name)',
    `name` VARCHAR(50) NOT NULL DEFAULT '0' COMMENT '帐号名称(Name on account)',
    `account_number` VARCHAR(50) NOT NULL DEFAULT '0' COMMENT '银行帐号(Account Number)',
    `branch_location` VARCHAR(255) NOT NULL DEFAULT '0' COMMENT '支行位置(Branch Location)',
    `description` VARCHAR(255) NOT NULL DEFAULT '0' COMMENT '银行描述',
    `status` ENUM('Y','N') NOT NULL DEFAULT 'N' COMMENT '银行帐号状态',
    `ctime` DATETIME NOT NULL COMMENT '创建时间',
    `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更改时间',
    PRIMARY KEY (`id`)
)
COMMENT='银行帐号'
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=2;

23.2.7. KEY

23.2.7.1. PRIMARY KEY

一般主键定义

PRIMARY KEY (`id`),

复合主键

PRIMARY KEY (`id`, `user_id`),

23.2.8. COMMENT

ALTER TABLE `neo`.`stuff` COMMENT = '用户表' ;
ALTER TABLE `neo`.`stuff` CHANGE COLUMN `name` `name` VARCHAR(50) NULL DEFAULT NULL COMMENT '姓名'  ;
ALTER TABLE `neo`.`stuff` CHANGE COLUMN `password` `password` VARCHAR(50) NULL DEFAULT NULL COMMENT '用户密码' ;

CREATE TABLE `stuff` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(50) DEFAULT NULL COMMENT ''姓名'',
  `password` varchar(50) DEFAULT NULL COMMENT ''用户密码'',
  `created` date NOT NULL DEFAULT ''0000-00-00'',
  PRIMARY KEY (`id`,`created`)
) ENGINE=MyISAM AUTO_INCREMENT=5 DEFAULT CHARSET=latin1 COMMENT=''用户表''
/*!50100 PARTITION BY HASH (year(created))
PARTITIONS 10 */


23.3. Index

23.3.1. SHOW INDEX

SHOW INDEX FROM tbl_name

垂直显示

SHOW INDEX FROM tbl_name\G

23.3.2. CREATE INDEX

CREATE INDEX index_name
ON table_name (column_name)

CREATE UNIQUE INDEX

CREATE UNIQUE INDEX index_name
ON table_name (column_name)

23.3.3. DROP INDEX

DROP INDEX index_name ON tbl_name

23.3.4. rebuild

SHOW INDEX FROM tbl_name
alter index IND_PK rebuild;


23.4. 外键(Foreign Key)

ON DELETE, ON UPDATE 事件触发限制，可选参数： RESTRICT | CASCADE | SET NULL | NO ACTION

  1. RESTRICT（限制外表中的外键改动）

  2. CASCADE（跟随外键改动）

  3. SET NULL（设空值）

  4. SET DEFAULT（设默认值）

  5. NO ACTION（无动作，默认的）

23.4.1. FOREIGN KEY (RESTRICT)

CREATE TABLE `bank_account_group_has_bank_account` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `bank_account_group_id` INT(10) UNSIGNED NOT NULL DEFAULT '0',
    `bank_account_id` INT(10) UNSIGNED NOT NULL DEFAULT '0',
    PRIMARY KEY (`id`),
    INDEX `FK_bank_account_group_has_bank_account_bank_account` (`bank_account_id`),
    INDEX `FK_bank_account_group_has_bank_account_bank_account_group` (`bank_account_group_id`),
    CONSTRAINT `FK_bank_account_group_has_bank_account_bank_account` FOREIGN KEY (`bank_account_id`) REFERENCES `bank_account` (`id`),
    CONSTRAINT `FK_bank_account_group_has_bank_account_bank_account_group` FOREIGN KEY (`bank_account_group_id`) REFERENCES `bank_account_group` (`id`)
)
COMMENT='bank_account_group 与 bank_account 的 N:M 关系'
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=35;


23.5. View

CREATE VIEW view_name AS
SELECT column_name(s)
FROM table_name
WHERE condition

update view

SQL CREATE OR REPLACE VIEW Syntax
CREATE OR REPLACE VIEW view_name AS
SELECT column_name(s)
FROM table_name
WHERE condition


23.6. Trigger

23.6.1. create trigger

history

CREATE TABLE user_history SELECT * FROM user WHERE 1 <> 1

DELIMITER //
CREATE TRIGGER user_history BEFORE update ON user FOR EACH ROW
BEGIN
insert into user_history SELECT * FROM user WHERE id = OLD.id;
END; //
DELIMITER ;

23.6.2. drop trigger

DROP TRIGGER admin_user_history;

DELIMITER //
CREATE TRIGGER admin_user_history BEFORE update ON admin_user FOR EACH ROW
BEGIN
insert into admin_user_history SELECT * FROM admin_user WHERE user_id = OLD.user_id;
END; //
DELIMITER;

23.6.3. show triggers

show triggers;

23.6.4. EXAMPLE

例 23.1. BEFORE/AFTER

DROP TRIGGER MY_TEST_MONITOR;
DELIMITER //
CREATE TRIGGER MY_TEST_MONITOR BEFORE insert ON MY_TEST FOR EACH ROW
BEGIN
    INSERT INTO MY_TEST_MONITOR SELECT * FROM MY_TEST WHERE TICKET = NEW.TICKET;
END; //
DELIMITER;

DROP TRIGGER MY_TEST_MONITOR;
DELIMITER //
CREATE TRIGGER MY_TEST_MONITOR AFTER insert ON MY_TEST FOR EACH ROW
BEGIN
    INSERT INTO MY_TEST_MONITOR SELECT * FROM MY_TEST WHERE TICKET = NEW.TICKET;
END; //
DELIMITER;

例 23.2. uuid()

delimiter $$

CREATE TABLE `member` (
  `uuid` char(36) NOT NULL,
  `username` varchar(20) DEFAULT NULL,
  `password` varchar(32) DEFAULT NULL,
  PRIMARY KEY (`uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8$$

CREATE
DEFINER=`root`@`%`
TRIGGER `test`.`member_before_insert`
BEFORE INSERT ON `test`.`member`
FOR EACH ROW
SET new.uuid = uuid()
$$


23.7. EVENT

23.7.1. 启用 EVENT

set GLOBAL event_scheduler=ON;

my.cnf 配置

event_scheduler=on

查看状态

mysql> select @@GLOBAL.event_scheduler;
+--------------------------+
| @@GLOBAL.event_scheduler |
+--------------------------+
| ON                       |
+--------------------------+
1 row in set (0.00 sec)

mysql> SHOW VARIABLES LIKE 'event_scheduler';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| event_scheduler | ON    |
+-----------------+-------+
1 row in set (0.01 sec)

23.7.2. 创建 EVENT

DROP EVENT IF EXISTS `captcha`;
DELIMITER //
CREATE DEFINER=`neo`@`%` EVENT `captcha` ON SCHEDULE EVERY 5 MINUTE STARTS '2013-07-08 16:27:03' ON COMPLETION PRESERVE ENABLE DO BEGIN
    delete from captcha where ctime < DATE_ADD(now(), INTERVAL -5 MINUTE);
END//
DELIMITER ;

23.7.3. 禁用/启用

ALTER EVENT captcha DISABLE;

ALTER EVENT captcha ENABLE;

23.7.4. show events

mysql> show events;
+--------+-------------+---------+-----------+-----------+------------+----------------+----------------+---------------------+------+---------+------------+----------------------+----------------------+--------------------+
| Db     | Name        | Definer | Time zone | Type      | Execute at | Interval value | Interval field | Starts              | Ends | Status  | Originator | character_set_client | collation_connection | Database Collation |
+--------+-------------+---------+-----------+-----------+------------+----------------+----------------+---------------------+------+---------+------------+----------------------+----------------------+--------------------+
| hx9999 | captcha     | neo@%   | SYSTEM    | RECURRING | NULL       | 5              | MINUTE         | 2013-07-08 16:27:03 | NULL | ENABLED |          1 | utf8                 | utf8_general_ci      | utf8_general_ci    |
| hx9999 | sms_ips_log | neo@%   | SYSTEM    | RECURRING | NULL       | '0 5'          | DAY_HOUR       | 2013-07-09 14:39:51 | NULL | ENABLED |          1 | utf8                 | utf8_general_ci      | utf8_general_ci    |
+--------+-------------+---------+-----------+-----------+------------+----------------+----------------+---------------------+------+---------+------------+----------------------+----------------------+--------------------+
2 rows in set (0.00 sec)

mysql> show events \G;
*************************** 1. row ***************************
                  Db: hx9999
                Name: captcha
             Definer: neo@%
           Time zone: SYSTEM
                Type: RECURRING
          Execute at: NULL
      Interval value: 5
      Interval field: MINUTE
              Starts: 2013-07-08 16:27:03
                Ends: NULL
              Status: ENABLED
          Originator: 1
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
*************************** 2. row ***************************
                  Db: hx9999
                Name: sms_ips_log
             Definer: neo@%
           Time zone: SYSTEM
                Type: RECURRING
          Execute at: NULL
      Interval value: '0 5'
      Interval field: DAY_HOUR
              Starts: 2013-07-09 14:39:51
                Ends: NULL
              Status: ENABLED
          Originator: 1
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
2 rows in set (0.00 sec)

ERROR:
No query specified


23.8. Partitioning

mysql> SHOW VARIABLES LIKE '%partition%';

+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| have_partitioning | YES   |
+-------------------+-------+
1 row in set (0.00 sec)

23.8.1. RANGE

18.5.1. Partitioning Keys, Primary Keys, and Unique Keys
This section discusses the relationship of partitioning keys with primary keys and unique keys. The rule governing this relationship can be expressed as follows: All columns used in the partitioning expression for a partitioned table must be part of every unique key that the table may have.

In other words, every unique key on the table must use every column in the table's partitioning expression. (This also includes the table's primary key, since it is by definition a unique key. This particular case is discussed later in this section.) For example, each of the following table creation statements is invalid:

SQL code:
mysql> create table tx (
    ->     id int not null ,
    ->     info_time date,
    ->     primary key(id,info_time)
    -> )
    -> PARTITION BY RANGE(info_time div 100)
    -> (
    ->     PARTITION p_2008_11 VALUES LESS THAN (200812),
    ->     PARTITION p_2008_12 VALUES LESS THAN (200901),
    ->     PARTITION p_2009_01 VALUES LESS THAN (200902),
    ->     PARTITION p_2009_02 VALUES LESS THAN (200903),
    ->     PARTITION p_2009_03 VALUES LESS THAN (200904),
    ->     PARTITION p_2009_04 VALUES LESS THAN (200905),
    ->     PARTITION p_catch_all VALUES LESS THAN MAXVALUE
    -> );
Query OK, 0 rows affected (0.17 sec)

mysql>

CREATE TABLE t1 (
    year_col  INT,
    some_data INT
)
PARTITION BY RANGE (year_col) (
    PARTITION p0 VALUES LESS THAN (1991),
    PARTITION p1 VALUES LESS THAN (1995),
    PARTITION p2 VALUES LESS THAN (1999),
    PARTITION p3 VALUES LESS THAN (2002),
    PARTITION p4 VALUES LESS THAN (2006),
    PARTITION p5 VALUES LESS THAN MAXVALUE
);

e.g.2

CREATE TABLE rc (
    a INT NOT NULL,
    b INT NOT NULL
)
PARTITION BY RANGE COLUMNS(a,b) (
    PARTITION p0 VALUES LESS THAN (10,5),
    PARTITION p1 VALUES LESS THAN (20,10),
    PARTITION p2 VALUES LESS THAN (MAXVALUE,15),
    PARTITION p3 VALUES LESS THAN (MAXVALUE,MAXVALUE)
);

CREATE TABLE part_tab
(
    c1 int default NULL,
    c2 varchar(30) default NULL,
    c3 date default NULL

) engine=myisam
PARTITION BY RANGE (year(c3)) (
      PARTITION p0 VALUES LESS THAN (2000) ,
      PARTITION p1 VALUES LESS THAN (2001) ,
      PARTITION p2 VALUES LESS THAN (2002) ,
      PARTITION p3 VALUES LESS THAN (2003) ,
      PARTITION p4 VALUES LESS THAN (2004) ,
      PARTITION p12 VALUES LESS THAN (2012),
      PARTITION p13 VALUES LESS THAN MAXVALUE
);

23.8.2. LIST

CREATE TABLE client_firms (
    id   INT,
    name VARCHAR(35)
)
PARTITION BY LIST (id) (
    PARTITION r0 VALUES IN (1, 5, 9, 13, 17, 21),
    PARTITION r1 VALUES IN (2, 6, 10, 14, 18, 22),
    PARTITION r2 VALUES IN (3, 7, 11, 15, 19, 23),
    PARTITION r3 VALUES IN (4, 8, 12, 16, 20, 24)
);

CREATE TABLE lc (
    a INT NULL,
    b INT NULL
)
PARTITION BY LIST COLUMNS(a,b) (
    PARTITION p0 VALUES IN( (0,0), (NULL,NULL) ),
    PARTITION p1 VALUES IN( (0,1), (0,2), (0,3), (1,1), (1,2) ),
    PARTITION p2 VALUES IN( (1,0), (2,0), (2,1), (3,0), (3,1) ),
    PARTITION p3 VALUES IN( (1,3), (2,2), (2,3), (3,2), (3,3) )
);

CREATE TABLE th (id INT, name VARCHAR(30), adate DATE)
PARTITION BY LIST(YEAR(adate))
(
  PARTITION p1999 VALUES IN (1995, 1999, 2003)
    DATA DIRECTORY = '/var/appdata/95/data'
    INDEX DIRECTORY = '/var/appdata/95/idx',
  PARTITION p2000 VALUES IN (1996, 2000, 2004)
    DATA DIRECTORY = '/var/appdata/96/data'
    INDEX DIRECTORY = '/var/appdata/96/idx',
  PARTITION p2001 VALUES IN (1997, 2001, 2005)
    DATA DIRECTORY = '/var/appdata/97/data'
    INDEX DIRECTORY = '/var/appdata/97/idx',
  PARTITION p2000 VALUES IN (1998, 2002, 2006)
    DATA DIRECTORY = '/var/appdata/98/data'
    INDEX DIRECTORY = '/var/appdata/98/idx'
);

23.8.3. HASH

CREATE TABLE `test` (
  `userid` int(10) unsigned NOT NULL auto_increment,
  `username` int(10) unsigned NOT NULL DEFAULT '0',
  `password` int(10) unsigned NOT NULL DEFAULT '0',

  primary key (`userid`),
  KEY `userid` (`username`)
) ENGINE=InnoDB
PARTITION BY HASH(userid)
PARTITIONS 8;

CREATE TABLE stuff (
    id INT AUTO_INCREMENT,
    name varchar(50),
    password varchar(50),
    created DATE,
    PRIMARY KEY (id, created)
)
PARTITION BY RANGE(year(created)) (
    PARTITION p0 VALUES LESS THAN (2010),
    PARTITION p1 VALUES LESS THAN (2012),
    PARTITION p2 VALUES LESS THAN MAXVALUE
);

更好的方法

CREATE TABLE stuff (
    id INT AUTO_INCREMENT,
    name varchar(50),
    password varchar(50),
    created DATE,
    PRIMARY KEY (id, created)
)
PARTITION BY HASH (year(created)) PARTITIONS 10;

我们演示一下

mysql> CREATE TABLE stuff (
    -> id INT AUTO_INCREMENT,
    -> name varchar(50),
    -> password varchar(50),
    -> created DATE,
    -> PRIMARY KEY (id, created)
    -> )
    -> PARTITION BY HASH (year(created)) PARTITIONS 10;
Query OK, 0 rows affected (0.08 sec)

mysql> insert into stuff (name,password,created) values('neo','test','2010-10-1');
Query OK, 1 row affected (0.06 sec)

mysql> insert into stuff (name,password,created) values('neo1','test','2012-2-1');
Query OK, 1 row affected (0.00 sec)

mysql> insert into stuff (name,password,created) values('neo2','test','2012-3-5');
Query OK, 1 row affected (0.00 sec)

mysql> insert into stuff (name,password,created) values('neo4','test','2011-1-5');
Query OK, 1 row affected (0.00 sec)

mysql> SELECT
    ->   partition_name part,
    ->   partition_expression expr,
    ->   partition_description descr,
    ->   table_rows
    -> FROM
    ->   INFORMATION_SCHEMA.partitions
    -> WHERE
    ->   TABLE_SCHEMA = schema()
    ->   AND TABLE_NAME='stuff';
+------+---------------+-------+------------+
| part | expr          | descr | table_rows |
+------+---------------+-------+------------+
| p0   | year(created) | NULL  |          1 |
| p1   | year(created) | NULL  |          1 |
| p2   | year(created) | NULL  |          2 |
| p3   | year(created) | NULL  |          0 |
| p4   | year(created) | NULL  |          0 |
| p5   | year(created) | NULL  |          0 |
| p6   | year(created) | NULL  |          0 |
| p7   | year(created) | NULL  |          0 |
| p8   | year(created) | NULL  |          0 |
| p9   | year(created) | NULL  |          0 |
+------+---------------+-------+------------+
10 rows in set (0.02 sec)

mysql> EXPLAIN PARTITIONS SELECT * FROM stuff WHERE created='2011-01-05'\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: stuff
   partitions: p1
         type: system
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 1
        Extra:
1 row in set (0.08 sec)

mysql> EXPLAIN PARTITIONS SELECT * FROM stuff WHERE created='2012-03-05'\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: stuff
   partitions: p2
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 2
        Extra: Using where
1 row in set (0.00 sec)

23.8.3.1. LINEAR HASH

CREATE TABLE employees (
    id INT NOT NULL,
    fname VARCHAR(30),
    lname VARCHAR(30),
    hired DATE NOT NULL DEFAULT '1970-01-01',
    separated DATE NOT NULL DEFAULT '9999-12-31',
    job_code INT,
    store_id INT
)
PARTITION BY LINEAR HASH( YEAR(hired) )
PARTITIONS 4;

23.8.4. KEY分区

按照KEY进行分区类似于按照HASH分区，除了HASH分区使用的用户定义的表达式，而KEY分区的 哈希函数是由MySQL 服务器提供。MySQL 簇（Cluster）使用函数MD5()来实现KEY分区；

CREATE TABLE tk (
    col1 INT NOT NULL,
    col2 CHAR(5),
    col3 DATE
)
PARTITION BY LINEAR KEY (col1)
PARTITIONS 3;

23.8.5. 分区管理

23.8.5.1. 新增分区

mysql 5.5+

为已经存在表添加分区

ALTER TABLE tbl_name  ADD PARTITION PARTITIONS 6;

新增 RANGE分区

ALTER TABLE category ADD PARTITION (PARTITION p4 VALUES IN (100,200,300,400)
                    DATA DIRECTORY = '/data/category'
                    INDEX DIRECTORY = '/data/category');

新增 LIST分区

CREATE TABLE expenses (
  expense_date DATE NOT NULL,
  category VARCHAR(30),
  amount DECIMAL (10,3)
);

ALTER TABLE expenses
PARTITION BY LIST COLUMNS (category)
(
  PARTITION p01 VALUES IN ( 'lodging', 'food'),
  PARTITION p02 VALUES IN ( 'flights', 'ground transportation'),
  PARTITION p03 VALUES IN ( 'leisure', 'customer entertainment'),
  PARTITION p04 VALUES IN ( 'communications'),
  PARTITION p05 VALUES IN ( 'fees')
);

新增 HASH分区

CREATE TABLE t1 (
    id INT,
    year_col INT
);

ALTER TABLE t1
    PARTITION BY HASH(id)
    PARTITIONS 8;

/* 在MySQL 5.1中*/
CREATE TABLE t2
(
  dt DATE
)
PARTITION BY RANGE (TO_DAYS(dt))
(
  PARTITION p01 VALUES LESS THAN (TO_DAYS('2007-01-01')),
  PARTITION p02 VALUES LESS THAN (TO_DAYS('2008-01-01')),
  PARTITION p03 VALUES LESS THAN (TO_DAYS('2009-01-01')),
  PARTITION p04 VALUES LESS THAN (MAXVALUE));

SHOW CREATE TABLE t2 \G
*************************** 1. row ***************************
       Table: t2
Create Table: CREATE TABLE `t2` (
  `dt` date DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/*!50100 PARTITION BY RANGE (TO_DAYS(dt))
(PARTITION p01 VALUES LESS THAN (733042) ENGINE = MyISAM,
 PARTITION p02 VALUES LESS THAN (733407) ENGINE = MyISAM,
 PARTITION p03 VALUES LESS THAN (733773) ENGINE = MyISAM,
 PARTITION p04 VALUES LESS THAN MAXVALUE ENGINE = MyISAM) */

 /*在MySQL 5.5中*/
CREATE TABLE t2
(
  dt DATE
)
PARTITION BY RANGE COLUMNS (dt)
(
  PARTITION p01 VALUES LESS THAN ('2007-01-01'),
  PARTITION p02 VALUES LESS THAN ('2008-01-01'),
  PARTITION p03 VALUES LESS THAN ('2009-01-01'),
  PARTITION p04 VALUES LESS THAN (MAXVALUE));

SHOW CREATE TABLE t2 \G
*************************** 1. row ***************************
       Table: t2
Create Table: CREATE TABLE `t2` (
  `dt` date DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/*!50500 PARTITION BY RANGE  COLUMNS(dt)
(PARTITION p01 VALUES LESS THAN ('2007-01-01') ENGINE = MyISAM,
 PARTITION p02 VALUES LESS THAN ('2008-01-01') ENGINE = MyISAM,
 PARTITION p03 VALUES LESS THAN ('2009-01-01') ENGINE = MyISAM,
 PARTITION p04 VALUES LESS THAN (MAXVALUE) ENGINE = MyISAM) */

23.8.5.2. 删除分区

删除分区 p0

ALERT TABLE users DROP PARTITION p0;

23.8.5.3. 重建分区

使用 REORGANIZE 重建分区。

RANGE 分区重建
ALTER TABLE users REORGANIZE PARTITION p0,p1 INTO (PARTITION p0 VALUES LESS THAN (6000000));

将原来的 p0,p1 分区合并起来，放到新的 p0 分区中。

LIST 分区重建
ALTER TABLE users REORGANIZE PARTITION p0,p1 INTO (PARTITION p0 VALUES IN(0,1,4,5,8,9,12,13));
将原来的 p0,p1 分区合并起来，放到新的 p0 分区中。

HASH/KEY 分区重建
ALTER TABLE users REORGANIZE PARTITION COALESCE PARTITION 2;
分区的数量改为2，
注意：在这里数量只能减少不能增加。想要增加可以用 ADD PARTITION 方法

调整HASH/KEY分区数量，将分区总数扩展到8个。

ALTER TABLE users ADD PARTITION PARTITIONS 8;

23.8.5.4. 分区维护

重建分区: 这和先删除保存在分区中的所有记录，然后重新插入它们，具有同样的效果。它可用于整理分区碎片。

示例：

ALTER TABLE t1 REBUILD PARTITION (p0, p1)；
·         优化分区：如果从分区中删除了大量的行，或者对一个带有可变长度的行（也就是说，有VARCHAR，BLOB，或TEXT类型的列）作了许多修改，可以使用“ALTER TABLE ... OPTIMIZE PARTITION”来收回没有使用的空间，并整理分区数据文件的碎片。

示例：

ALTER TABLE t1 OPTIMIZE PARTITION (p0, p1)；
在一个给定的分区表上使用“OPTIMIZE PARTITION”等同于在那个分区上运行CHECK PARTITION，ANALYZE PARTITION，和REPAIR PARTITION。

·         分析分区：读取并保存分区的键分布。

示例：

ALTER TABLE t1 ANALYZE PARTITION (p3)；
·         修补分区： 修补被破坏的分区。

示例：

ALTER TABLE t1 REPAIR PARTITION (p0,p1);
·         检查分区： 可以使用几乎与对非分区表使用CHECK TABLE 相同的方式检查分区。

示例：

ALTER TABLE trb3 CHECK PARTITION (p1)；

23.8.6. EXPLAIN PARTITIONS

EXPLAIN PARTITIONS

mysql> EXPLAIN PARTITIONS SELECT * FROM users\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: users
   partitions: p0,p1,p2,p3,p4,p5,p6
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 7
        Extra:
1 row in set (0.03 sec)

mysql> EXPLAIN PARTITIONS SELECT * FROM users WHERE id < 5\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: users
   partitions: p0,p1,p2,p3,p4,p5,p6
         type: range
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 4
          ref: NULL
         rows: 7
        Extra: Using where
1 row in set (0.00 sec)

23.8.7. SHOW CREATE TABLE

SHOW CREATE TABLE

mysql> SHOW CREATE TABLE users\G
*************************** 1. row ***************************
       Table: users
Create Table: CREATE TABLE `users` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(20) NOT NULL DEFAULT '',
  `birthday` datetime DEFAULT NULL,
  PRIMARY KEY (`id`,`username`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1
/*!50100 PARTITION BY KEY (id,username)
PARTITIONS 7 */
1 row in set (0.00 sec)

23.8.8. INFORMATION_SCHEMA.partitions 表

SELECT
  partition_name part,
  partition_expression expr,
  partition_description descr,
  table_rows
FROM
  INFORMATION_SCHEMA.partitions
WHERE
  TABLE_SCHEMA = schema()
  AND TABLE_NAME='employees';

select
  partition_name part,
  partition_expression expr,
  from_seconds(partition_description) descr,
  table_rows
FROM
INFORMATION_SCHEMA.partitions
WHERE
    TABLE_SCHEMA = 'test'
    AND TABLE_NAME='t2';

23.8.9. ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function

http://dev.mysql.com/doc/refman/5.1/en/partitioning-limitations-partitioning-keys-unique-keys.html


23.9. Engine

mysql> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| FEDERATED          | NO      | Federated MySQL storage engine                                 | NULL         | NULL | NULL       |
| MRG_MYISAM         | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| MyISAM             | YES     | MyISAM storage engine                                          | NO           | NO   | NO         |
| BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| CSV                | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| ARCHIVE            | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| PERFORMANCE_SCHEMA | YES     | Performance Schema                                             | NO           | NO   | NO         |
| InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)

修改与切换引擎

ALTER TABLE `test` ENGINE=BLACKHOLE;
ALTER TABLE `test` ENGINE=InnoDB;

23.9.1. FEDERATED

启用 FEDERATED 引擎, 服务器环境 Ubuntu 13.04

$ sudo vim /etc/mysql/conf.d/federated.cnf
[mysqld]
federated

$ sudo service mysql restart

mysql> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| FEDERATED          | YES     | Federated MySQL storage engine                                 | NO           | NO   | NO         |
| MRG_MYISAM         | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| MyISAM             | YES     | MyISAM storage engine                                          | NO           | NO   | NO         |
| BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| CSV                | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| ARCHIVE            | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| PERFORMANCE_SCHEMA | YES     | Performance Schema                                             | NO           | NO   | NO         |
| InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)

A 服务器

CREATE TABLE `t1` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `name` VARCHAR(50) NOT NULL,
    `sex` ENUM('Y','N') NULL DEFAULT 'Y',
    `passwd` VARCHAR(50) NULL DEFAULT NULL,
    `ctime` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
    `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (`id`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=4;

B 服务器

DROP TABLE `users`;

CREATE TABLE `users` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `name` VARCHAR(50) NOT NULL,
    `sex` ENUM('Y','N') NULL DEFAULT 'Y',
    `ctime` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
    `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (`id`)
) ENGINE=FEDERATED connection = 'mysql://www:qwer123@192.168.2.1:3306/test/t1';

上面字段描述是你需要的字段，并非所有字段。这里屏蔽了passwd字段

提示

connection = 'mysql://用户名:密码@主机:端口/数据库/表名'

mysql> DROP TABLE `users`;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE TABLE `users` (
    -> `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    -> `name` VARCHAR(50) NOT NULL,
    -> `sex` ENUM('Y','N') NULL DEFAULT 'Y',
    -> `ctime` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
    -> `mtime` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    -> PRIMARY KEY (`id`)
    -> ) ENGINE=FEDERATED connection = 'mysql://www:qwer123@192.168.2.1:3306/test/t1';
Query OK, 0 rows affected (0.06 sec)

mysql>
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| users          |
+----------------+
1 row in set (0.00 sec)

mysql> desc users;
+-------+------------------+------+-----+---------------------+-----------------------------+
| Field | Type             | Null | Key | Default             | Extra                       |
+-------+------------------+------+-----+---------------------+-----------------------------+
| id    | int(10) unsigned | NO   | PRI | NULL                | auto_increment              |
| name  | varchar(50)      | NO   |     | NULL                |                             |
| sex   | enum('Y','N')    | YES  |     | Y                   |                             |
| ctime | timestamp        | NO   |     | 0000-00-00 00:00:00 |                             |
| mtime | timestamp        | NO   |     | CURRENT_TIMESTAMP   | on update CURRENT_TIMESTAMP |
+-------+------------------+------+-----+---------------------+-----------------------------+
5 rows in set (0.00 sec)

mysql> select * from users;
+----+------+------+---------------------+---------------------+
| id | name | sex  | ctime               | mtime               |
+----+------+------+---------------------+---------------------+
|  1 | neo  | Y    | 0000-00-00 00:00:00 | 2013-05-17 18:05:09 |
|  2 | zen  | Y    | 0000-00-00 00:00:00 | 2013-05-17 18:05:11 |
|  3 | lily | N    | 0000-00-00 00:00:00 | 2013-05-17 18:05:22 |
+----+------+------+---------------------+---------------------+
3 rows in set (0.01 sec)

FEDERATED 与 mysqldump 问题！

切记，mysqldump 只会dump出使用FEDERATED引擎表的结构,不会包含数据。

23.9.2. BLACKHOLE

CREATE TABLE test(id INT, val CHAR(10)) ENGINE = BLACKHOLE;


第 24 章 DML (Data Manipulation Language)

目录

24.1. INSERT
24.1.1. INSERT INTO ... SELECT
SELECT - retrieve data from the a database
INSERT - insert data into a table
UPDATE - updates existing data within a table
DELETE - deletes all records from a table, the space for the records remain
CALL - call a PL/SQL or Java subprogram
EXPLAIN PLAN - explain access path to data
LOCK TABLE - control concurrency

24.1. INSERT

24.1.1. INSERT INTO ... SELECT

SET @OLDTMP_SQL_MODE=@@SQL_MODE, SQL_MODE='';
DELIMITER //
CREATE TRIGGER `members_mobile_insert` BEFORE INSERT ON `members_mobile` FOR EACH ROW BEGIN
    insert into members_location(id,province,city) select NEW.id,mobile_location.province,mobile_location.city from  mobile_location where mobile_location.id = md5(LEFT(NEW.number, 7));
END//
DELIMITER ;
SET SQL_MODE=@OLDTMP_SQL_MODE;


第 25 章 SQL Statement Syntax

Structured Query Language

目录

25.1. DISTINCT
25.2. group by
25.3. replace
25.4. OUTFILE/LOAD DATA INFILE
25.4.1. Export data to CSV from MySQL
25.5. SQL 92
25.6. Functions and Operators
25.6.1. COUNT
25.6.2. UUID()
25.6.3. String
25.6.3.1. LEFT/RIGHT
25.6.3.2. RPAD/LPAD
25.6.3.3. CONCAT
25.6.3.4. GROUP_CONCAT
25.6.3.5. replace
25.6.4. Date and Time
25.6.4.1. year/month/day hour:minite:second
25.6.4.2. Unix time
25.6.4.3. DATE_FORMAT
25.6.4.4. DATE_SUB/DATE_ADD
25.6.4.5. datediff / timediff
25.7. 事务处理和锁定语句
25.7.1. 排他锁
25.7.2. 表的加锁与解锁
25.7.3. 事务所用到的表
25.8. MySQL 专有命令
25.8.1. SQL_NO_CACHE

25.1. DISTINCT

SELECT DISTINCT user.name FROM user

SELECT DISTINCT user.name FROM user


25.2. group by

统计重复的手机号吗

select * from (select count(mobile) as c, mobile from member where length(mobile) >= 11 group by mobile) as m where m.c > 1;


25.3. replace

replace into

replace into (id) value('1')


25.4. OUTFILE/LOAD DATA INFILE

查询结果输出到文件

SELECT * FROM tablename INTO OUTFILE '/tmp/tablename.txt';

使用tee将屏幕输出到文件

mysql>tee /home/neo/screen.txt
mysql>select * from user;
mysql>exit

SELECT * INTO OUTFILE '/home/mark/Orders.txt'
    FIELDS
    TERMINATED BY = ','
    FROM Orders
    WHERE Order_Date >= '2000-01-01'

LOAD DATA INFILE 'data.txt' INTO TABLE db2.my_table;

25.4.1. Export data to CSV from MySQL

SELECT *
INTO OUTFILE '/tmp/products.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM products


25.5. SQL 92

insert + select

insert into product_type_commission select id,5,1,1,0,0,0,0,0,0 from product_type where title='notebook' and is_physical=0;

update table1,table2

begin;
ALTER TABLE `customer` ADD COLUMN `cutoff_time` TIMESTAMP NOT NULL default '0000-00-00 00:00:00';
update customer,agent set customer.cutoff_time = agent.cutoff_time where customer.id = agent.id;
ALTER TABLE `agent` DROP COLUMN `cutoff_time`;
commit;

update table1 set field1 = (select value from table2)

UPDATE
    transaction
SET
    transaction.total_sold_price = (
        SELECT
            SUM(transaction_item.price)
    FROM
            transaction_item
        WHERE transaction_item.transaction_id = 100
    )
WHERE
    transaction.id = 100

update table1, (select * from other) as table2 set table1.field1 = table2.field1

UPDATE
    transaction,(	SELECT
                        SUM(product_item.bought_price) AS total_bought_price, transaction_item.transaction_id
                    FROM
                        transaction_item
                    WHERE
                        transaction_item.transaction_id IN ( '123','456' )
                 ) as total
SET
    transaction.total_bought_price = total.total_bought_price
WHERE
    transaction.id = total.transaction_id

join + subquery

select u.*,t.category,t.items,t.[property] from tb_sysregchkusers as u left join (select a.items as category, b.* from (select id, items from tb_sysregchktask where categoryid=0) as a left join tb_sysregchktask as b on b.categoryid=a.id ) as t on u.taskID=t.id

select * from tb_sysregchklog where CONVERT(datetime,CONVERT(varchar(10),checkTime,120)) between convert(datetime,'2007-12-12') and convert(datetime,'2007-12-12')

select DISTINCT user_point_history.user_id,user.username,
(select count(id) from transaction where id = user_point_history.transaction_id) as transactions,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type != 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_earned,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type = 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_redeemed
from user_point_history,user where user_point_history.user_id = user.id;

(total_points_earned - total_points_redeemed) as current_balance_points

select user_id, username, transactions, total_points_earned, total_points_redeemed, (total_points_earned - total_points_redeemed) as current_balance_points
from (select DISTINCT user_point_history.user_id,user.username,
(select count(id) from transaction where id = user_point_history.transaction_id) as transactions,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type != 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_earned,
(SELECT SUM(u_p_h.points) FROM user_point_history as u_p_h WHERE u_p_h.type = 'RDMP' AND u_p_h.status IN('pr','ac') AND u_p_h.user_id = user_point_history.user_id) as total_points_redeemed
from user_point_history,user where user_point_history.user_id = user.id) as user_performance;

subquery作为一个字段使用

select product_type_attribute.*,(select 'selected' from product_type_attribute_set where product_type_attribute_set.product_type_attribute_id = product_type_attribute.id and product_type_attribute_set.product_type_id = 26) as selected
from product_type_attribute;


25.6. Functions and Operators

25.6.1. COUNT

count()

SELECT (SELECT count(1) FROM ecs_category) as 'Export category count',
    (SELECT count(1) FROM ecs_goods) as 'Goods count',
    (SELECT count(1) FROM ecs_goods_attr) as 'Attr count';

25.6.2. UUID()

SELECT UUID(),LENGTH(UUID()),UUID_SHORT(), LENGTH(UUID_SHORT());

25.6.3. String

25.6.3.1. LEFT/RIGHT

LEFT(str,len)

mysql> select left(concat('1','0000000'),5) as number;
+--------+
| number |
+--------+
| 10000  |
+--------+
1 row in set (0.00 sec)

RIGHT(str,len)

mysql> select right(concat('0000000','1'),5) as number;
+--------+
| number |
+--------+
| 00001  |
+--------+
1 row in set (0.00 sec)

25.6.3.2. RPAD/LPAD

补齐长度用'0'填充

RPAD(str,len,padstr)
mysql> select rpad('10',5,'0') as txt;
+-------+
| txt   |
+-------+
| 10000 |
+-------+
1 row in set (0.01 sec)
LPAD(str,len,padstr)
mysql> select lpad('10',5,'0') as txt;
+-------+
| txt   |
+-------+
| 00010 |
+-------+
1 row in set (0.00 sec)

25.6.3.3. CONCAT

CONCAT(str1,str2,...)

mysql> select concat('Neo',' ','Chen') as Name;
+----------+
| Name     |
+----------+
| Neo Chen |
+----------+
1 row in set (0.00 sec)

25.6.3.4. GROUP_CONCAT

mysql> select GROUP_CONCAT(CONVERT( username , CHAR (16)) order by username desc) as username from test;
+-------------------------------------------+
| username                                  |
+-------------------------------------------+
| jam,jam2,john,john2,john3,neo,neo1,neo2   |
+-------------------------------------------+
6 rows in set, 1 warning (0.01 sec)

25.6.3.5. replace

select replace(goods_desc,':8000','') from ecs_goods;

update ecs_goods set goods_desc=replace(goods_desc,':8000','');

25.6.4. Date and Time

25.6.4.1. year/month/day hour:minite:second

mysql> select year('2012-03-20');
+--------------------+
| year('2012-03-20') |
+--------------------+
|               2012 |
+--------------------+
1 row in set (0.00 sec)

mysql> select month('2012-03-20');
+---------------------+
| month('2012-03-20') |
+---------------------+
|                   3 |
+---------------------+
1 row in set (0.00 sec)

mysql> select day('2012-03-20');
+-------------------+
| day('2012-03-20') |
+-------------------+
|                20 |
+-------------------+
1 row in set (0.00 sec)

mysql> select hour('12:30:55');
+------------------+
| hour('12:30:55') |
+------------------+
|               12 |
+------------------+
1 row in set (0.00 sec)

mysql> select minute('12:30:55');
+--------------------+
| minute('12:30:55') |
+--------------------+
|                 30 |
+--------------------+
1 row in set (0.00 sec)

mysql> select second('12:30:55');
+--------------------+
| second('12:30:55') |
+--------------------+
|                 55 |
+--------------------+
1 row in set (0.00 sec)

25.6.4.2. Unix time

mysql> SELECT UNIX_TIMESTAMP('2005-03-27 02:00:00');
+---------------------------------------+
| UNIX_TIMESTAMP('2005-03-27 02:00:00') |
+---------------------------------------+
|                            1111885200 |
+---------------------------------------+
mysql> SELECT FROM_UNIXTIME(1111885200);
+---------------------------+
| FROM_UNIXTIME(1111885200) |
+---------------------------+
| 2005-03-27 03:00:00       |
+---------------------------+

SELECT UNIX_TIMESTAMP('2012-01-01 00:00:00');
SELECT UNIX_TIMESTAMP('2012-07-30 00:00:00');
SELECT UNIX_TIMESTAMP();
SELECT UNIX_TIMESTAMP('2009-08-06') ;
SELECT UNIX_TIMESTAMP( curdate( ) );

select FROM_UNIXTIME(UNIX_TIMESTAMP('2012-07-30 00:00:00'), '%Y-%m-%d');
SELECT FROM_UNIXTIME( 1249488000, '%Y年%m月%d日' );

select FROM_UNIXTIME(createtime, '%m') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2012-01-01 00:00:00') and UNIX_TIMESTAMP('2012-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m');
select FROM_UNIXTIME(createtime, '%m') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2011-01-01 00:00:00') and UNIX_TIMESTAMP('2011-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m');

select FROM_UNIXTIME(createtime, '%m-%d') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2011-01-01 00:00:00') and UNIX_TIMESTAMP('2011-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m-%d');
select FROM_UNIXTIME(createtime, '%m-%d') as month, count(1) as count from members where createtime BETWEEN UNIX_TIMESTAMP('2012-01-01 00:00:00') and UNIX_TIMESTAMP('2012-12-31 00:00:00') group by FROM_UNIXTIME(createtime, '%m-%d');

25.6.4.3. DATE_FORMAT

DATE_FORMAT() 函数用于以不同的格式显示日期/时间数据。

语法
DATE_FORMAT(date,format)
date 参数是合法的日期。format 规定日期/时间的输出格式。

可以使用的格式有：
格式	描述
%a	缩写星期名
%b	缩写月名
%c	月，数值
%D	带有英文前缀的月中的天
%d	月的天，数值(00-31)
%e	月的天，数值(0-31)
%f	微秒
%H	小时 (00-23)
%h	小时 (01-12)
%I	小时 (01-12)
%i	分钟，数值(00-59)
%j	年的天 (001-366)
%k	小时 (0-23)
%l	小时 (1-12)
%M	月名
%m	月，数值(00-12)
%p	AM 或 PM
%r	时间，12-小时（hh:mm:ss AM 或 PM）
%S	秒(00-59)
%s	秒(00-59)
%T	时间, 24-小时 (hh:mm:ss)
%U	周 (00-53) 星期日是一周的第一天
%u	周 (00-53) 星期一是一周的第一天
%V	周 (01-53) 星期日是一周的第一天，与 %X 使用
%v	周 (01-53) 星期一是一周的第一天，与 %x 使用
%W	星期名
%w	周的天 （0=星期日, 6=星期六）
%X	年，其中的星期日是周的第一天，4 位，与 %V 使用
%x	年，其中的星期一是周的第一天，4 位，与 %v 使用
%Y	年，4 位
%y	年，2 位

实例

下面的脚本使用 DATE_FORMAT() 函数来显示不同的格式。我们使用 NOW() 来获得当前的日期/时间：
DATE_FORMAT(NOW(),'%b %d %Y %h:%i %p')
DATE_FORMAT(NOW(),'%m-%d-%Y')
DATE_FORMAT(NOW(),'%d %b %y')
DATE_FORMAT(NOW(),'%d %b %Y %T:%f')

SELECT DATE_FORMAT(NOW(),'%Y-%m-%d');

select DATE_FORMAT(asctime,'%Y-%m-%d') as Date, count(1) as Count from logging where tag='www' and facility='login' group by DATE_FORMAT(asctime,'%Y-%m-%d') order by asctime desc;

25.6.4.4. DATE_SUB/DATE_ADD

当前时间向后推10天

mysql> select DATE_SUB(now(), INTERVAL 240 HOUR);
+------------------------------------+
| DATE_SUB(now(), INTERVAL 240 HOUR) |
+------------------------------------+
| 2012-03-09 10:26:03                |
+------------------------------------+
1 row in set (0.00 sec)

mysql> select DATE_SUB(now(), INTERVAL 24 HOUR);
+-----------------------------------+
| DATE_SUB(now(), INTERVAL 24 HOUR) |
+-----------------------------------+
| 2012-03-18 10:28:43               |
+-----------------------------------+
1 row in set (0.00 sec)

DELETE from Message where created < DATE_sub(now(), INTERVAL 240 HOUR);

select * from PRICES_HISTORY where DATE_FORMAT(TIME ,GET_FORMAT(DATE,'ISO')) = (
select if ( WEEKDAY(CURRENT_DATE())=6 , DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) , CURRENT_DATE())
)
25.6.4.4.1. DATE_ADD
SELECT DATE_ADD('1998-01-02', INTERVAL 31 DAY);

25.6.4.5. datediff / timediff

计算时间差，两个时间相减结果

mysql> select timediff('22:20:00','17:30:00');
+---------------------------------+
| timediff('22:20:00','17:30:00') |
+---------------------------------+
| 04:50:00                        |
+---------------------------------+
1 row in set (0.00 sec)

mysql> select datediff('2008-08-08 12:00:00', '2008-08-01 00:00:00');
+--------------------------------------------------------+
| datediff('2008-08-08 12:00:00', '2008-08-01 00:00:00') |
+--------------------------------------------------------+
|                                                      7 |
+--------------------------------------------------------+
1 row in set (0.00 sec)


25.7. 事务处理和锁定语句

Transactional and Locking Statements

开始事务 begin、start transaction 或者 set autocommit=0

事务的特征：原子性（Atomiocity）、一致性（Consistency）、隔离性（Isolation）和持久性（Durability），这四个特性简称ACID特性。
    原子性：事务是数据库的逻辑工作单位，事务中包括的所有操作要么都做，要么都不做。
    一致性：事务执行的结果必须是使数据库从一个一致性的状态变到另外一个一致性状态。
    隔离性：一个事务的执行不能被其他事务干扰。即一个事务内部的操作及使用的数据对其他事务是隔离的，并发执行的各个事务之间互相不干扰。
    持久性：一个事务一旦成功提交，对数据库中数据的修改就是持久性的。接下来其他的其他操作或故障不应该对其执行结果有任何影响。

事务隔离模式

     1) READ UNCOMMITED
    SELECT的时候允许脏读，即SELECT会读取其他事务修改而还没有提交的数据。

    2)READ COMMITED
    SELECT的时候无法重复读，即同一个事务中两次执行同样的查询语句，若在第一次与第二次查询之间时间段，其他事务又刚好修改了其查询的数据且提交了，则两次读到的数据不一致。

    3) REPEATABLE READ
    SELECT的时候可以重复读，即同一个事务中两次执行同样的查询语句，得到的数据始终都是一致的。实现的原理是，在一个事务对数据行执行读取或写入操作时锁定了这些数据行。
    但是这种方式又引发了幻想读的问题。因为只能锁定读取或写入的行，不能阻止另一个事务插入数据，后期执行同样的查询会产生更多的结果。

    4)SERIALIZABLE
    与可重复读的唯一区别是，默认把普通的SELECT语句改成SELECT …. LOCK IN SHARE MODE。即为查询语句涉及到的数据加上共享琐，阻塞其他事务修改真实数据。
    serializable模式中，事务被强制为依次执行。这是SQL标准建议的默认行为。

  查看InnoDB系统级别的事务隔离级别：mysql> SELECT @@global.tx_isolation;
  查看InnoDB会话级别的事务隔离级别：mysql> SELECT @@tx_isolation;
  修改InnoDB系统级别的事务隔离级别：mysql> set global transaction isolation level read committed;
  修改InnoDB会话级别的事务隔离级别：mysql> set session transaction isolation level read committed;

锁机制

    1) 共享锁：由读表操作加上的锁，加锁后其他用户只能获取该表或行的共享锁，不能获取排它锁，也就是说只能读不能写
    2) 排它锁：由写表操作加上的锁，加锁后其他用户不能获取该表或行的任何锁，典型是mysql事务中的

    锁的范围:
    行锁: 对某行记录加上锁
    表锁: 对整个表加上锁

共享锁(share mode), 排他锁(for update)

25.7.1. 排他锁

下面做作一个实验，验证锁的效果

终端一,首先进入事务状态然后运行下面语句

mysql> begin;
Query OK, 0 rows affected (0.00 sec)

mysql> select * from t1 where id='3' for update;
+----+--------+---------------------+---------------------+
| id | name   | ctime               | mtime               |
+----+--------+---------------------+---------------------+
|  3 | test   | 0000-00-00 00:00:00 | 2013-01-14 13:05:41 |
+----+--------+---------------------+---------------------+
1 row in set (0.00 sec)

终端二, 查询表中数据

mysql> select * from t1;
+----+--------+---------------------+---------------------+
| id | name   | ctime               | mtime               |
+----+--------+---------------------+---------------------+
|  1 | neo    | 0000-00-00 00:00:00 | 2013-01-14 13:00:00 |
|  2 | zen    | 0000-00-00 00:00:00 | 2013-01-14 13:00:43 |
|  3 | test   | 0000-00-00 00:00:00 | 2013-01-14 13:05:41 |
+----+--------+---------------------+---------------------+
3 rows in set (0.00 sec)

增加“for update”查询非锁定记录

mysql> select * from t1 where id=2 for update;
+----+------+---------------------+---------------------+
| id | name | ctime               | mtime               |
+----+------+---------------------+---------------------+
|  2 | zen  | 0000-00-00 00:00:00 | 2013-01-14 13:00:43 |
+----+------+---------------------+---------------------+
1 row in set (0.00 sec)

查询被锁定记录

mysql> select * from t1 where id=3 for update;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

查询所有记录，因为记录中包含了id=3那条，所以也不允许查询。

mysql> select * from t1 for update;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

测试修改记录

mysql> UPDATE `t1` SET `name`='testaa' WHERE  `id`=3;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

提示

在没有出现ERROR 1205 (HY000)的这段时间，只要终端一中执行commit,rollback锁就释放了.终端二中的语句就会运行。

select trx_query from information_schema.innodb_trx; 可以查看被锁的SQL语句

25.7.2. 表的加锁与解锁

LOCK TABLES tablename WRITE;
LOCK TABLES tablename READ;

...
...

UNLOCK TABLES;

25.7.3. 事务所用到的表

information_schema

select * from innodb_trx;
select * from innodb_lock_waits;
select * from innodb_locks;


25.8. MySQL 专有命令

25.8.1. SQL_NO_CACHE

SELECT /*!40001 SQL_NO_CACHE */ * FROM table


第 26 章 PostgreSQL GUI

目录

26.1. pgAdmin III
26.2. phpPgAdmin
26.3. Monitoring
26.3.1. PgBadger
26.4. pgModeler - PostgreSQL Database Modeler

26.1. pgAdmin III

http://www.pgadmin.org/


26.2. phpPgAdmin

homepage: http://phppgadmin.sourceforge.net/

$ wget http://nchc.dl.sourceforge.net/sourceforge/phppgadmin/phpPgAdmin-4.2.2.tar.bz2
$ tar jxvf phpPgAdmin-4.2.2.tar.bz2
$ ln -s phpPgAdmin-4.2.2 phpPgAdmin


26.3. Monitoring

26.3.1. PgBadger

http://projects.dalibo.org/pgbadger/example.html


26.4. pgModeler - PostgreSQL Database Modeler

http://pgmodeler.com.br/


第 27 章 Barman Backup & recovery for PostgreSQL

http://www.pgbarman.org/


第 28 章 pgbouncer - lightweight connection pooler for PostgreSQL

目录

28.1. 安装 pgbouncer
28.1.1. Ubuntu
28.1.2. CentOS
28.2. 配置 pgbouncer

http://pgfoundry.org/projects/pgbouncer/

28.1. 安装 pgbouncer

28.1.1. Ubuntu

Ubuntu 13.04

$ apt-cache search pgbouncer
pgbouncer - lightweight connection pooler for PostgreSQL

$ sudo apt-get install pgbouncer

$ dpkg -L pgbouncer
/.
/usr
/usr/share
/usr/share/man
/usr/share/man/man5
/usr/share/man/man5/pgbouncer.5.gz
/usr/share/man/man1
/usr/share/man/man1/pgbouncer.1.gz
/usr/share/doc
/usr/share/doc/pgbouncer
/usr/share/doc/pgbouncer/config.html
/usr/share/doc/pgbouncer/README
/usr/share/doc/pgbouncer/README.Debian
/usr/share/doc/pgbouncer/README.html
/usr/share/doc/pgbouncer/faq.html
/usr/share/doc/pgbouncer/copyright
/usr/share/doc/pgbouncer/todo.html
/usr/share/doc/pgbouncer/examples
/usr/share/doc/pgbouncer/examples/pgbouncer.ini.gz
/usr/share/doc/pgbouncer/examples/userlist.txt
/usr/share/doc/pgbouncer/NEWS.gz
/usr/share/doc/pgbouncer/AUTHORS
/usr/share/doc/pgbouncer/usage.html
/usr/share/doc/pgbouncer/changelog.Debian.gz
/usr/sbin
/usr/sbin/pgbouncer
/etc
/etc/init.d
/etc/init.d/pgbouncer
/etc/pgbouncer
/etc/pgbouncer/userlist.txt
/etc/pgbouncer/pgbouncer.ini
/etc/default
/etc/default/pgbouncer

28.1.2. CentOS

# yum install pgbouncer

# rpm -ql pgbouncer.x86_64 0:1.5.4-1.rhel6
/etc/pgbouncer/mkauth.py
/etc/pgbouncer/mkauth.pyc
/etc/pgbouncer/mkauth.pyo
/etc/pgbouncer/pgbouncer.ini
/etc/rc.d/init.d/pgbouncer
/etc/sysconfig/pgbouncer
/usr/bin/pgbouncer
/usr/share/doc/pgbouncer-1.5.4
/usr/share/doc/pgbouncer-1.5.4/AUTHORS
/usr/share/doc/pgbouncer-1.5.4/NEWS
/usr/share/doc/pgbouncer-1.5.4/README
/usr/share/man/man1/pgbouncer.1.gz
/usr/share/man/man5/pgbouncer.5.gz


28.2. 配置 pgbouncer

databases 配置

[databases]
main = host=localhost port=5432 dbname=mydb user=myuser password=mypass connect_query='SELECT 1'

pgbouncer

[pgbouncer]
logfile = /var/log/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = 127.0.0.1
listen_port = 6432
auth_type = trust
auth_file = /etc/pgbouncer/userlist.txt
admin_users = postgres
stats_users = stats, postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 100
default_pool_size = 20

例 28.1. /etc/pgbouncer/pgbouncer.ini

# grep -v '^;' /etc/pgbouncer/pgbouncer.ini | grep -v '^$'

[databases]
main = host=localhost port=5432 dbname=test user=test password=test connect_query='SELECT 1'
[pgbouncer]
logfile = /var/log/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = 127.0.0.1
listen_port = 1521
auth_type = trust
auth_file = /etc/pgbouncer/userlist.txt
admin_users = postgres
stats_users = stats, postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 100
default_pool_size = 20


第 29 章 Foreign data wrappers

http://wiki.postgresql.org/wiki/Foreign_data_wrappers


第 30 章 FAQ

30.1. tcpip_socket?
30.1. | tcpip_socket?
 | 8.1.5之后版本不在使用tcpip_socket,改用listen_addresses


部分 IV. Oracle

目录

31. Installing Oracle Database
31.1. Installing Oracle Database 10g Release 2 on Linux x86
31.2. 11gR2
31.3. Installing Oracle Client
31.4. Silence Install - Database
31.4.1. Response File
31.4.2. OS 配置脚本
31.4.3. 运行 runInstaller
31.5. Silence Install - Client
32. Manager
32.1. listener.ora
32.1.1. TNS 配置
32.1.1.1. 11gR2
32.2. Account
32.2.1. show user
32.2.2. DEFAULT_TABLESPACE
32.2.3. unlock/lock
32.3. 显示表
32.4. oracle 817 script
32.5. Script for automatic startup on boot
32.6. Run level shell script to start Oracle 10g services on RedHat Enterprise Linux (RHAS 4)
33. Oracle Gui
33.1. Create instance
33.2. Oracle Net Configuration Assistant
33.3. Oracle Enterprise Manager
33.4. Other GUI - phpOraAdmin
34. CLI
34.1. SQL*Plus
34.1.1. startup/shutdown
34.1.2. conn
34.1.3. parameter
34.1.3.1. db
34.1.3.2. instance_name
34.1.3.3. service_name
34.1.3.4. global_name
34.1.3.5. db_name
34.1.3.6. db_domain
34.1.3.7. sga
34.1.3.8. size
34.1.3.9. spfile
34.1.3.10. cache
34.1.4. $ORACLE_HOME/sqlplus/admin/glogin.sql
34.1.5. @运行SQL
34.2. lsnrctl
34.3. RMAN
34.3.1. 数据库模式
34.3.2. 完全备份
34.3.3. 增量备份
34.3.4. 恢复数据库
34.3.5. 是用tar打包rman文件


第 31 章 Installing Oracle Database

目录

31.1. Installing Oracle Database 10g Release 2 on Linux x86
31.2. 11gR2
31.3. Installing Oracle Client
31.4. Silence Install - Database
31.4.1. Response File
31.4.2. OS 配置脚本
31.4.3. 运行 runInstaller
31.5. Silence Install - Client

31.1. Installing Oracle Database 10g Release 2 on Linux x86

reference: http://www.oracle.com/technology/pub/articles/smiley_10gdb_install.html

To make these changes, cut and paste the following commands as root:

过程 31.1. Configure linux step by step

  1. Verifying Your Installation

    rpm -q binutils compat-db control-center gcc gcc-c++ glibc glibc-common \
gnome-libs libstdc++ libstdc++-devel make pdksh sysstat xscreensaver libaio openmotif21

    installing package

    yum install compat-gcc-32 compat-gcc-32-c++ compat-gcc-32-g77 compat-libf2c-32 compat-libstdc++-296 compat-libstdc++-33 compat-db compat-readline43

  2. Verifying System Requirements

    grep MemTotal /proc/meminfo
grep SwapTotal /proc/meminfo

    Swap = mem * 2

  3. Create the Oracle Groups and User Account

    groupadd oinstall
groupadd dba
useradd -m -g oinstall -G dba oracle
passwd oracle
id oracle

  4. Create Directories

    mkdir -p /u01/app/oracle
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle

  5. Configuring the Linux Kernel Parameters

cat >> /etc/sysctl.conf <<EOF
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=262144
net.core.wmem_max=262144
EOF
/sbin/sysctl -p

    Run the following commands as root to verify your settings:

    /sbin/sysctl -a | grep shm
/sbin/sysctl -a | grep sem
/sbin/sysctl -a | grep file-max
/sbin/sysctl -a | grep ip_local_port_range
/sbin/sysctl -a | grep rmem_default
/sbin/sysctl -a | grep rmem_max
/sbin/sysctl -a | grep wmem_default
/sbin/sysctl -a | grep wmem_max

  6. Setting Shell Limits for the oracle User

cat >> /etc/security/limits.conf <<EOF
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
EOF

  7. /etc/profile

cat >> /etc/profile <<EOF
if [ \$USER = "oracle" ]; then
 if [ \$SHELL = "/bin/ksh" ]; then
 ulimit -p 16384
 ulimit -n 65536
 else
 ulimit -u 16384 -n 65536
 fi
 umask 022
fi
EOF

cat >> /etc/csh.login <<EOF
if ( \$USER == "oracle" ) then
 limit maxproc 16384
 limit descriptors 65536
 umask 022
endif
EOF

  8. .bash_profile

    # su - oracle
$ vim .bash_profile

export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0.1
export ORACLE_SID=orcl
export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib

过程 31.2. Installing Oracle

  1. 编辑 /10201_database_linux32/database/install/oraparam.ini 添加

    vim 10201_database_linux32/database/install/oraparam.ini

### #[Certified Versions]

Linux=redhat-3,SuSE-9,redhat-4,centos-5,UnitedLinux-1.0,asianux-1,asianux-2

[Linux-centos-5.1-optional]
TEMP_SPACE=80
SWAP_SPACE=150
MIN_DISPLAY_COLORS=256

  2. install

gunzip xxxx.cpio.gz
cpio -idmv < xxxx.cpio
export LANG=en_US
./runInstaller

  3. dbstart

    # su - oracle
# dbstart

提示打开 /ade/vikrkuma_new/oracle/bin/tnslsnr 失败

    编辑 /u01/app/oracle/product/10.2.0.1/bin/dbstart

    # Set this to bring up Oracle Net Listener
ORACLE_HOME_LISTNER=/ade/vikrkuma_new/oracle

应该是在78行，将其改为：

# Set this to bring up Oracle Net Listener
ORACLE_HOME_LISTNER=$ORACLE_HOME

    /etc/oratable 将最后一行的最后一个字符由 “N” 改为 “Y”

    orcl: /u01/app/oracle/product/10.2.0.1:Y

过程 31.3. Configuring Storage

  1. Partition the Disks

    fdisk -l /dev/sdb

  2. Filesystems

    ZFS or btrfs

  3. Create the Mount Point

mkdir /u02

    Add the New Filesystem to /etc/fstab

/dev/sdb1 /u02 xfs defaults 1 1

    Mount the New Filesystem

mount /u02
df -h /u02

  4. Create Oracle Directories and Set Permissions

mkdir -p /u02/oradata/demo1
chown -R oracle:oinstall /u02/oradata
chmod -R 775 /u02/oradata

  5. Create a New Tablespace in the New Filesystem

Ex:
$ sqlplus

SQL*Plus: Release 10.2.0.1.0 - Production on Sun Nov 27 15:50:50 2005

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

Enter user-name: system
Enter password:

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

SQL> create tablespace data1
  2  datafile '/u02/oradata/demo1/data1_01.dbf' size 100m
  3  extent management local
  4  segment space management auto;

Tablespace created.

Now you can use the new tablespace to store database objects such as tables and indexes.

Ex:
SQL> create table demotab (id number(5) not null primary key,
 2 name varchar2(50) not null,
 3 amount number(9,2))
 4 tablespace data1;

Table created.


31.2. 11gR2

unzip linux.x64_11gR2_database_1of2.zip
unzip linux.x64_11gR2_database_2of2.zip

[oracle@wcs ~]$ vi .bash_profile
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_BASE=/opt/oracle/
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
export ORACLE_SID=orcl
export ORACLE_TERM=xterm
export ORACLE_HOME_LISTNER=$ORACLE_HOME
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/usr/X11R6/lib64/
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export LD_ASSUME_KERNEL=2.6.18
export NLS_LANG="AMERICAN_AMERICA.ZHS16GBK"

cat >> /etc/sysctl.conf <<EOF
fs.aio-max-nr = 3145728
fs.file-max = 6815744
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
kernel.shmmni = 4096
kernel.sem = 250 32000 100 142
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
EOF
/sbin/sysctl -p

切换到root用户运行下面脚本

/opt/oracle/oraInventory/orainstRoot.sh
/opt/oracle/product/11.2.0.1/client/root.sh

[root@oracle ~]# /opt/oracle/oraInventory/orainstRoot.sh
Changing permissions of /opt/oracle/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oracle/oraInventory to oinstall.
The execution of the script is complete.

[root@oracle ~]# /opt/oracle/product/11.2.0/dbhome_1/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /opt/oracle/product/11.2.0/dbhome_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.


31.3. Installing Oracle Client

orains.sh

#!/bin/bash
groupadd oinstall
groupadd dba
useradd -m -g oinstall -G dba oracle
echo "oracle:oracle" | chpasswd
id oracle
mkdir -p /opt/oracle
chown oracle.oinstall /opt/oracle

cat >> /etc/sysctl.conf <<EOF
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 32768 61000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_max=262144
EOF
/sbin/sysctl -p

Run the following commands as root to verify your settings:

/sbin/sysctl -a | grep shm
/sbin/sysctl -a | grep sem
/sbin/sysctl -a | grep file-max
/sbin/sysctl -a | grep ip_local_port_range
/sbin/sysctl -a | grep rmem_default
/sbin/sysctl -a | grep rmem_max
/sbin/sysctl -a | grep wmem_default
/sbin/sysctl -a | grep wmem_max

export TMP=/tmp
export TMPDIR=/tmp
export ORACLE_BASE=/opt/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0.1/client
export PATH=$ORACLE_HOME/bin:$PATH

yum install sysstat libaio-devel glibc-devel elfutils-libelf-devel unixODBC unixODBC-devel

unzip linux.x64_11gR2_client.zip

[oracle@wcs ~]$ cd client/
[oracle@wcs client]$ ./runInstaller

登录到root用户运行下面脚本

/opt/oracle/oraInventory/orainstRoot.sh
/opt/oracle/product/11.2.0.1/client/root.sh

[root@wcs oracle]# /opt/oracle/oraInventory/orainstRoot.sh
Changing permissions of /opt/oracle/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oracle/oraInventory to oinstall.
The execution of the script is complete.
[root@wcs oracle]# /opt/oracle/product/11.2.0.1/client/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /opt/oracle/product/11.2.0.1/client

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.

至此，Oracle客户端安装完毕


31.4. Silence Install - Database

31.4.1. Response File

创建response文件 /home/oracle/db.rsp

[oracle@oracle ~]$ cat db.rsp

#-------------------------------------------------------------------------------
# Do not change the following system generated value.
#-------------------------------------------------------------------------------
oracle.install.responseFileVersion=/oracle/install/rspfmt_dbinstall_response_schema_v11_2_0

#-------------------------------------------------------------------------------
# The installation option can be one of the following
# 1. INSTALL_DB_SWONLY
# 2. INSTALL_DB_AND_CONFIG
# 3. UPGRADE_DB
#-------------------------------------------------------------------------------
oracle.install.option=INSTALL_DB_AND_CONFIG

#-------------------------------------------------------------------------------
# This variable holds the hostname of the system as set by the user.
# It can be used to force the installation to use an alternative
# hostname rather than using the first hostname found on the system
# (e.g., for systems with multiple hostnames and network interfaces).
#-------------------------------------------------------------------------------
ORACLE_HOSTNAME=oracle.example.com

#-------------------------------------------------------------------------------
# Unix group to be set for the inventory directory.
#-------------------------------------------------------------------------------
UNIX_GROUP_NAME=oinstall

#-------------------------------------------------------------------------------
# Inventory location.
#-------------------------------------------------------------------------------
INVENTORY_LOCATION=/opt/oracle/oraInventory

#-------------------------------------------------------------------------------
# Specify the languages in which the components will be installed.
#
# en   : English                  ja   : Japanese
# fr   : French                   ko   : Korean
# ar   : Arabic                   es   : Latin American Spanish
# bn   : Bengali                  lv   : Latvian
# pt_BR: Brazilian Portuguese     lt   : Lithuanian
# bg   : Bulgarian                ms   : Malay
# fr_CA: Canadian French          es_MX: Mexican Spanish
# ca   : Catalan                  no   : Norwegian
# hr   : Croatian                 pl   : Polish
# cs   : Czech                    pt   : Portuguese
# da   : Danish                   ro   : Romanian
# nl   : Dutch                    ru   : Russian
# ar_EG: Egyptian                 zh_CN: Simplified Chinese
# en_GB: English (Great Britain)  sk   : Slovak
# et   : Estonian                 sl   : Slovenian
# fi   : Finnish                  es_ES: Spanish
# de   : German                   sv   : Swedish
# el   : Greek                    th   : Thai
# iw   : Hebrew                   zh_TW: Traditional Chinese
# hu   : Hungarian                tr   : Turkish
# is   : Icelandic                uk   : Ukrainian
# in   : Indonesian               vi   : Vietnamese
# it   : Italian
#
# Example : SELECTED_LANGUAGES=en,fr,ja
#-------------------------------------------------------------------------------
SELECTED_LANGUAGES=en

#-------------------------------------------------------------------------------
# Complete path of the Oracle Home
#-------------------------------------------------------------------------------
ORACLE_HOME=/opt/oracle/product/11.2.0/dbhome_1

#-------------------------------------------------------------------------------
# Complete path of the Oracle Base.
#-------------------------------------------------------------------------------
ORACLE_BASE=/opt/oracle

#-------------------------------------------------------------------------------
# Installation Edition of the component.
#
# The value should contain only one of these choices.
# EE     : Enterprise Edition
# SE     : Standard Edition
# SEONE  : Standard Edition One
# PE     : Personal Edition (WINDOWS ONLY)
#-------------------------------------------------------------------------------
oracle.install.db.InstallEdition=EE

#-------------------------------------------------------------------------------
# This property is considered only if InstallEdition is EE.
#
# true  : Components mentioned as part of 'customComponents' property
#         are considered for install.
# false : Value for 'customComponents' is not considered.
#-------------------------------------------------------------------------------
oracle.install.db.isCustomInstall=false

#-------------------------------------------------------------------------------
# This property is considered only if 'IsCustomInstall' is set to true
#
# Description: List of Enterprise Edition Options you would like to install.
#
#              The following choices are available. You may specify any
#              combination of these choices.  The components you choose should
#              be specified in the form "internal-component-name:version"
#              Below is a list of components you may specify to install.
#
#              oracle.oraolap:11.2.0.0.2 - Oracle OLAP
#              oracle.rdbms.dm:11.2.0.0.2 - Oracle Data Mining RDBMS Files
#              oracle.rdbms.dv:11.2.0.0.2 - Oracle Database Vault option
#              oracle.rdbms.lbac:11.2.0.0.2 - Oracle Label Security
#              oracle.rdbms.partitioning:11.2.0.0.2 - Oracle Partitioning
#              oracle.rdbms.rat:11.2.0.0.2 - Oracle Real Application Testing
#              oracle.clrintg.ode_net:11.2.0.0.2 - Oracle Database Extensions for .NET 1.x (Windows)
#              oracle.clrintg.ode_net_2:11.2.0.0.2 - Oracle Database Extensions for .NET 2.0 (Windows)
#-------------------------------------------------------------------------------
oracle.install.db.customComponents=

#-------------------------------------------------------------------------------
oracle.install.db.DBA_GROUP=dba

#-------------------------------------------------------------------------------
oracle.install.db.OPER_GROUP=dba

#-------------------------------------------------------------------------------
# This variable represents the cluster node names selected by the .
# user for installation
#-------------------------------------------------------------------------------
oracle.install.db.CLUSTER_NODES=

#-------------------------------------------------------------------------------
# One of the following
# - GENERAL_PURPOSE
# - TRANSACTION_PROCESSING
# - DATAWAREHOUSE
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.type=GENERAL_PURPOSE

#-------------------------------------------------------------------------------
# Global Database Name
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.globalDBName=orcl.example.com

#-------------------------------------------------------------------------------
# The Starter Database SID
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.SID=orcl

#-------------------------------------------------------------------------------
# Database character set
#
#  One of the following
#  AL32UTF8, WE8ISO8859P15, WE8MSWIN1252, EE8ISO8859P2,
#  EE8MSWIN1250, NE8ISO8859P10, NEE8ISO8859P4, BLT8MSWIN1257,
#  BLT8ISO8859P13, CL8ISO8859P5, CL8MSWIN1251, AR8ISO8859P6,
#  AR8MSWIN1256, EL8ISO8859P7, EL8MSWIN1253, IW8ISO8859P8,
#  IW8MSWIN1255, JA16EUC, JA16EUCTILDE, JA16SJIS, JA16SJISTILDE,
#  KO16MSWIN949, ZHS16GBK, TH8TISASCII, ZHT32EUC, ZHT16MSWIN950,
#  ZHT16HKSCS, WE8ISO8859P9, TR8MSWIN1254, VN8MSWIN1258
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.characterSet=AL32UTF8
#-------------------------------------------------------------------------------
# Specify the total memory allocation for the database. (in MB)
# Value should be at least 256 MB, and should not exceed the
# total physical memory available on the system.
# Example: oracle.install.db.config.starterdb.memoryLimit=40
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.memoryLimit=6218
oracle.install.db.config.starterdb.memoryOption=true

#-------------------------------------------------------------------------------
# This variable controls whether to load Example Schemas onto
# the starter database or not.
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.installExampleSchemas=true

#-------------------------------------------------------------------------------
# This include enabling audit settings, configuring password
# profiles and revoking some grants to public. These settings
# are provided by default.  You may choose to disable all.
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.enableSecuritySettings=true

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.password.ALL=

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.password.SYS=

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.password.SYSTEM=

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.password.SYSMAN=

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.password.DBSNMP=

#-------------------------------------------------------------------------------
# Can be one of the following
# 1. GRID_CONTROL
# 2. DB_CONTROL
#
oracle.install.db.config.starterdb.control=DB_CONTROL

#-------------------------------------------------------------------------------
# Determines the Management Service to use if Grid Control
# is selected to manage the database.
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.gridcontrol.gridControlServiceURL=

#-------------------------------------------------------------------------------
# Determines whether to receive email notification for
# critical alerts when using DB control.
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.dbcontrol.enableEmailNotification=false

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.dbcontrol.emailAddress=

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.dbcontrol.SMTPServer=

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.automatedBackup.enable=false

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.automatedBackup.osuid=

#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.automatedBackup.ospwd=

#-------------------------------------------------------------------------------
# Can be one of the following
# - FILE_SYSTEM_STORAGE
# - ASM_STORAGE
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.storageType=FILE_SYSTEM_STORAGE

#-------------------------------------------------------------------------------
# Database file location:
# directory for datafiles, control files, redo logs.
#
# Applicable only when oracle.install.db.config.starterdb.storage=FILE_SYSTEM_STORAGE
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.fileSystemStorage.dataLocation=/opt/oracle/oradata

#-------------------------------------------------------------------------------
# Backup and recovery location
#
# Applicable only when oracle.install.db.config.starterdb.storage=FILE_SYSTEM_STORAGE
#-------------------------------------------------------------------------------
oracle.install.db.config.starterdb.fileSystemStorage.recoveryLocation=

#-------------------------------------------------------------------------------
# Name of ASM disk group to be used for storage.
#
# Applicable only when oracle.install.db.config.starterdb.storageType=ASM_STORAGE
#-------------------------------------------------------------------------------
oracle.install.db.config.asm.diskGroup=

#-------------------------------------------------------------------------------
# Password for ASMSNMP user of the ASM instance.
#
# Applicable only when oracle.install.db.config.starterdb.storage=ASM_STORAGE
#-------------------------------------------------------------------------------
oracle.install.db.config.asm.ASMSNMPPassword=

#------------------------------------------------------------------------------
# Specify the My Oracle Support Account Username.
#
#  Example   : MYORACLESUPPORT_USERNAME=metalink
#------------------------------------------------------------------------------
MYORACLESUPPORT_USERNAME=neo.chen@msn.com

#------------------------------------------------------------------------------
# Specify the My Oracle Support Account Username password.
#
# Example    : MYORACLESUPPORT_PASSWORD=password
#------------------------------------------------------------------------------
MYORACLESUPPORT_PASSWORD=

#------------------------------------------------------------------------------
# Specify whether to enable the user to set the password for
# My Oracle Support credentials. The value can be either true or false.
# If left blank it will be assumed to be false.
#
# Example    : SECURITY_UPDATES_VIA_MYORACLESUPPORT=true
#------------------------------------------------------------------------------
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false

#------------------------------------------------------------------------------
# Specify whether user wants to give any proxy details for connection.
# The value can be either true or false. If left blank it will be assumed
# to be false.
#
# Example    : DECLINE_SECURITY_UPDATES=false
#------------------------------------------------------------------------------
DECLINE_SECURITY_UPDATES=false

#------------------------------------------------------------------------------
# Specify the Proxy server name. Length should be greater than zero.
#
# Example    : PROXY_HOST=proxy.domain.com
#------------------------------------------------------------------------------
PROXY_HOST=

#------------------------------------------------------------------------------
# Specify the proxy port number. Should be Numeric and atleast 2 chars.
#
# Example    : PROXY_PORT=25
#------------------------------------------------------------------------------
PROXY_PORT=

31.4.2. OS 配置脚本

/home/oracle/orains.sh

#!/bin/bash
ORACLE_BASE=/opt/oracle
ORACLE_PASSWORD="oracle"

groupadd oinstall
groupadd dba
useradd -m -g oinstall -G dba oracle
echo "oracle:$ORACLE_PASSWORD" | chpasswd

mkdir -p $ORACLE_BASE
chown -R oracle:oinstall $ORACLE_BASE
chmod -R 775 $ORACLE_BASE

cat >> /etc/sysctl.conf <<EOF
fs.aio-max-nr = 3145728
fs.file-max = 6815744
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
kernel.shmmni = 4096
kernel.sem = 250 32000 100 142
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
EOF

cat >> /etc/security/limits.conf <<EOF
oracle soft nproc 2048
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
EOF

cat >> /home/oracle/.bash_profile <<\EOF
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=$(hostname)
export ORACLE_BASE=/opt/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
export ORACLE_SID=orcl
export ORACLE_TERM=xterm
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib64:/usr/lib64:/usr/local/lib64
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export LD_ASSUME_KERNEL=2.6.18
export NLS_LANG="american_america.UTF8"
export NLS_LANG="AMERICAN_AMERICA.US7ASCII"
#export NLS_LANG="AMERICAN_AMERICA.ZHS16GBK"
#export NLS_LANG="SIMPLIFIED CHINESE_CHINA.ZHS16GBK"
#export NLS_LANG="TRADITIONAL CHINESE_TAIWAN.ZHT16MSWIN950"
#export NLS_LANG="JAPANESE_JAPAN.WE8MSWIN1252"
EOF

cat >> /home/oracle/.bashrc <<\EOF
alias sysdba='sqlplus "/ as sysdba"'
EOF

cat >> /etc/oraInst.loc <<EOF
inventory_loc=$ORACLE_BASE/oraInventory
inst_group=oinstall
EOF
chmod 664 /etc/oraInst.loc

cat >> /etc/yum.repos.d/rhel-source-local.repo <<\EOF
[rhel-source-local]
name=Red Hat Enterprise Linux $releasever - Source
baseurl=file:///media/cdrom0/Server
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
EOF

yum install gcc gcc-c++ glibc-devel libstdc++ libstdc++-devel libaio-devel sysstat libaio-devel elfutils-libelf-devel unixODBC unixODBC-devel

su - root
/home/oracle/orains.sh

31.4.3. 运行 runInstaller

unzip linux.x64_11gR2_database_1of2.zip
unzip linux.x64_11gR2_database_2of2.zip
cd database
./runInstaller -silent -noconfig -responseFile /home/oracle/db.rsp

检查安装日志

ls /opt/oracle/oraInventory/logs/


31.5. Silence Install - Client

###############################################################################
## Copyright(c) Oracle Corporation 1998,2008. All rights reserved.           ##
##                                                                           ##
## Specify values for the variables listed below to customize                ##
## your installation.                                                        ##
##                                                                           ##
## Each variable is associated with a comment. The comment                   ##
## can help to populate the variables with the appropriate                   ##
## values.							             ##
##                                                                           ##
###############################################################################

#-------------------------------------------------------------------------------
# Do not change the following system generated value.
#-------------------------------------------------------------------------------
oracle.install.responseFileVersion=http://www.oracle.com/2007/install/rspfmt_clientinstall_response_schema_v11_2_0

#-------------------------------------------------------------------------------
# This variable holds the hostname of the system as set by the user.
# It can be used to force the installation to use an alternative
# hostname rather than using the first hostname found on the system
# (e.g., for systems with multiple hostnames and network interfaces).
ORACLE_HOSTNAME=wcs.example.com
#-------------------------------------------------------------------------------
# Unix group to be set for the inventory directory.
UNIX_GROUP_NAME=oinstall
#-------------------------------------------------------------------------------
# Inventory location.
INVENTORY_LOCATION=/opt/oracle/oraInventory
#-------------------------------------------------------------------------------
# Specify the languages in which the components will be installed.
#
# en   : English                  ja   : Japanese
# fr   : French                   ko   : Korean
# ar   : Arabic                   es   : Latin American Spanish
# bn   : Bengali                  lv   : Latvian
# pt_BR: Brazilian Portuguese     lt   : Lithuanian
# bg   : Bulgarian                ms   : Malay
# fr_CA: Canadian French          es_MX: Mexican Spanish
# ca   : Catalan                  no   : Norwegian
# hr   : Croatian                 pl   : Polish
# cs   : Czech                    pt   : Portuguese
# da   : Danish                   ro   : Romanian
# nl   : Dutch                    ru   : Russian
# ar_EG: Egyptian                 zh_CN: Simplified Chinese
# en_GB: English (Great Britain)  sk   : Slovak
# et   : Estonian                 sl   : Slovenian
# fi   : Finnish                  es_ES: Spanish
# de   : German                   sv   : Swedish
# el   : Greek                    th   : Thai
# iw   : Hebrew                   zh_TW: Traditional Chinese
# hu   : Hungarian                tr   : Turkish
# is   : Icelandic                uk   : Ukrainian
# in   : Indonesian               vi   : Vietnamese
# it   : Italian
#
# Example : SELECTED_LANGUAGES=en,fr,ja
SELECTED_LANGUAGES=en
#-------------------------------------------------------------------------------
# Complete path of the Oracle Home
ORACLE_HOME=/opt/oracle/product/11.2.0.1/client
#-------------------------------------------------------------------------------
# Complete path of the Oracle Base.
ORACLE_BASE=/opt/oracle
#------------------------------------------------------------------------------
#Name       : INSTALL_TYPE
#Datatype   : String
#Description: Installation type of the component.
#
#             The following choices are available. The value should contain
#             only one of these choices.
#             InstantClient : InstantClient
#             Administrator : Administrator
#             Runtime       : Runtime
#             Custom        : Custom
#
#Example    : INSTALL_TYPE = "Administrator"
#------------------------------------------------------------------------------
oracle.install.client.installType=Administrator
#-------------------------------------------------------------------------------
# Name       : oracle.install.client.customComponents
# Datatype   : StringList
#
# This property is considered only if INSTALL_TYPE is set to "Custom"
#
# Description: List of Client Components you would like to install
#
#   The following choices are available. You may specify any
#   combination of these choices.  The components you choose should
#   be specified in the form "internal-component-name:version"
#   Below is a list of components you may specify to install.
#
# oracle.sqlj:11.2.0.1.0 -- "Oracle SQLJ"
# oracle.rdbms.util:11.2.0.1.0 -- "Oracle Database Utilities"
# oracle.javavm.client:11.2.0.1.0 -- "Oracle Java Client"
# oracle.sqlplus:11.2.0.1.0 -- "SQL*Plus"
# oracle.dbjava.jdbc:11.2.0.1.0 -- "Oracle JDBC/THIN Interfaces"
# oracle.ldap.client:11.2.0.1.0 -- "Oracle Internet Directory Client"
# oracle.rdbms.oci:11.2.0.1.0 -- "Oracle Call Interface (OCI)"
# oracle.precomp:11.2.0.1.0 -- "Oracle Programmer"
# oracle.xdk:11.2.0.1.0 -- "Oracle XML Development Kit"
# oracle.network.aso:11.2.0.1.0 -- "Oracle Advanced Security"
# oracle.assistants.oemlt:11.2.0.1.0 -- "Enterprise Manager Minimal Integration"
# oracle.oraolap.mgmt:11.2.0.1.0 -- "OLAP Analytic Workspace Manager and Worksheet"
# oracle.network.client:11.2.0.1.0 -- "Oracle Net"
# oracle.ordim.client:11.2.0.1.0 -- "Oracle Multimedia Client Option"
# oracle.ons:11.2.0.0.0 -- "Oracle Notification Service"
# oracle.odbc:11.2.0.1.0 -- "Oracle ODBC Driver"
# oracle.has.client:11.2.0.1.0 -- "Oracle Clusterware High Availability API"
# oracle.dbdev:11.2.0.1.0 -- "Oracle SQL Developer"
# oracle.rdbms.scheduler:11.2.0.1.0 -- "Oracle Scheduler Agent"
#
# Example    : oracle.install.client.customComponents="oracle.precomp:11.2.0.1.0","oracle.ons:11.2.0.0.0","oracle.oraolap.mgmt:11.2.0.1.0","oracle.rdbms.scheduler:11.2.0.1.0"
#-------------------------------------------------------------------------------
oracle.install.client.customComponents=
#-------------------------------------------------------------------------------
#Name       : MTS_PORT
#Datatype   : int
#Description: Port number to be used for by the Oracle MTS Recovery Service to listen
#		  for requests. This needs to be entered in case oracle.ntoramts is
#	        selected in the list of custom components in custom install
#
#
#Example    : MTS_PORT = 2030
#------------------------------------------------------------------------------
oracle.install.client.oramtsPortNumber=

#------------------------------------------------------------------------------
# Host name to be used for by the Oracle Scheduler Agent.
# This needs to be entered in case oracle.rdbms.scheduler is selected in the
# list of custom components during custom install
#
# Example    : oracle.install.client.schedulerAgentHostName = acme.domain.com
#------------------------------------------------------------------------------
oracle.install.client.schedulerAgentHostName=

#------------------------------------------------------------------------------
# Port number to be used for by the Oracle Scheduler Agent.
# This needs to be entered in case oracle.rdbms.scheduler is selected in the
# list of custom components during custom install
#
# Example: oracle.install.client.schedulerAgentPortNumber = 1500
#------------------------------------------------------------------------------
oracle.install.client.schedulerAgentPortNumber=

#!/bin/bash
groupadd oinstall
groupadd dba
useradd -m -g oinstall -G dba oracle
echo "oracle:oracle" | chpasswd
id oracle

mkdir -p /opt/oracle
chown oracle.oinstall /opt/oracle

cat >> /etc/sysctl.conf <<EOF
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 32768 61000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_max=262144
EOF
/sbin/sysctl -p

cat >> /home/oracle/.bash_profile <<\EOF
export TMP=/tmp
export TMPDIR=/tmp
export ORACLE_BASE=/opt/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0.1/client
export PATH=$ORACLE_HOME/bin:$PATH
EOF

cat >> /home/oracle/.bashrc <<\EOF
alias sysdba='sqlplus "/ as sysdba"'
EOF

yum install sysstat libaio-devel glibc-devel elfutils-libelf-devel unixODBC unixODBC-devel

unzip linux.x64_11gR2_client.zip

cd client/
./runInstaller -silent -noconfig -responseFile /home/oracle/client.rsp

# /opt/oracle/oraInventory/orainstRoot.sh
# /opt/oracle/product/11.2.0.1/client/root.sh


第 32 章 Manager

目录

32.1. listener.ora
32.1.1. TNS 配置
32.1.1.1. 11gR2
32.2. Account
32.2.1. show user
32.2.2. DEFAULT_TABLESPACE
32.2.3. unlock/lock
32.3. 显示表
32.4. oracle 817 script
32.5. Script for automatic startup on boot
32.6. Run level shell script to start Oracle 10g services on RedHat Enterprise Linux (RHAS 4)

32.1. listener.ora

[root@database ~]# cat /u01/app/oracle/product/10.2.0.1/network/admin/listener.ora
# listener.ora Network Configuration File: /u01/app/oracle/product/10.2.0.1/network/admin/listener.ora
# Generated by Oracle configuration tools.

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = /u01/app/oracle/product/10.2.0.1)
      (PROGRAM = extproc)
    )
  )

LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1))
      (ADDRESS = (PROTOCOL = TCP)(HOST = database.example.com)(PORT = 1521))
    )
  )

32.1.1. TNS 配置

tnsnames.ora 文件默认在 network/admin/tnsnames.ora

有些情况如你没有权限修改network/admin/tnsnames.ora, 你可以在$HOME下创建一个.tnsnames.ora文件

oradb10g =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = db1.domain.com)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SID = oradb10g)
    )
  )

oradb =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = db2.domain.com)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SID = oradb)
    )
  )

测试TNS

$ sqlplus user@oradb

32.1.1.1. 11gR2

ORCL =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.0.5)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = orcl.example.com)
    )
  )

SERVICE_NAME通過show parameter service_name;查詢


32.2. Account

select username,account_status from dba_users;

32.2.1. show user

SQL> show user;
USER is "SYS"

SQL> select user from dual;

USER
------------------------------
SYS

32.2.2. DEFAULT_TABLESPACE

用户默认表空间

SQL> SELECT DEFAULT_TABLESPACE FROM DBA_USERS WHERE USERNAME='WCUSER';

DEFAULT_TABLESPACE
------------------------------
WCSDB

SQL> SELECT DEFAULT_TABLESPACE FROM DBA_USERS WHERE USERNAME=(select user from dual);

DEFAULT_TABLESPACE
------------------------------
SYSTEM

32.2.3. unlock/lock

帐号加锁与解锁

alter user scott account unlock /lock;

SQL> alter user scott account unlock;

User altered.
SQL> select username,account_status from dba_users where username='SCOTT';

USERNAME                       ACCOUNT_STATUS
------------------------------ --------------------------------
SCOTT                          EXPIRED

SQL> alter user scott account lock;

User altered.

SQL> select username,account_status from dba_users where username='SCOTT';

USERNAME                       ACCOUNT_STATUS
------------------------------ --------------------------------
SCOTT                          EXPIRED & LOCKED

SQL>


32.3. 显示表

select * from tab where tabtype='SYNONYM';

select name,type,referenced_name from user_dependencies;


32.4. oracle 817 script

Oracle 817 数据库启动脚本

#!/bin/bash
##############################################################
# Script to startup and shutdown Oracle and listener
# File:/etc/rc.d/init.d/orcl
##############################################################
# Setup environment for script execution
#./home/oracle/.bash_profile
#
ORACLE_HOME=/u01/app/oracle/product/8.1.7
# Determine and execute action based on command line parameter
case "$1" in
start)
echo "Starting Oracle database(s) listed in /etc/oratab ..."
sleep 2
su - oracle -c "$ORACLE_HOME/bin/dbstart"
echo "Starting TNS listener ..."
sleep 2
su - oracle -c "$ORACLE_HOME/bin/lsnrctl start"
touch /var/lock/subsys/orcl
;;
stop)
echo "Shutting down TNS listener ..."
sleep 2
su - oracle -c "$ORACLE_HOME/bin/lsnrctl stop"
echo "Shutting down Oracle database(s) listed in /etc/oratab ..."
su - oracle -c "$ORACLE_HOME/bin/dbshut"
rm -f /var/lock/subsys/orcl
;;
status)
ps -ax | grep -e ora_ -e tnslsnr
;;
*)
echo "Usage: $1 {start|stop|status}"
;;
esac
exit 0


32.5. Script for automatic startup on boot

#!/bin/bash
#
# Run-level Startup script for the Oracle Instance and Listener
#
# chkconfig: 345 91 19
# description: Startup/Shutdown Oracle listener and instance

ORA_HOME="/u01/app/oracle/product/9.2.0.1.0"
ORA_OWNR="oracle"

# if the executables do not exist -- display error

if [ ! -f $ORA_HOME/bin/dbstart -o ! -d $ORA_HOME ]
then
        echo "Oracle startup: cannot start"
        exit 1
fi

# depending on parameter -- startup, shutdown, restart
# of the instance and listener or usage display

case "$1" in
    start)
        # Oracle listener and instance startup
        echo -n "Starting Oracle: "
        su - $ORA_OWNR -c "$ORA_HOME/bin/lsnrctl start"
        su - $ORA_OWNR -c $ORA_HOME/bin/dbstart
        touch /var/lock/subsys/oracle
        echo "OK"
        ;;
    stop)
        # Oracle listener and instance shutdown
        echo -n "Shutdown Oracle: "
        su - $ORA_OWNR -c "$ORA_HOME/bin/lsnrctl stop"
        su - $ORA_OWNR -c $ORA_HOME/bin/dbshut
        rm -f /var/lock/subsys/oracle
        echo "OK"
        ;;
    reload|restart)
        $0 stop
        $0 start
        ;;
    *)
        echo "Usage: $0 start|stop|restart|reload"
        exit 1
esac
exit 0


32.6. Run level shell script to start Oracle 10g services on RedHat Enterprise Linux (RHAS 4)

#!/bin/bash
##############################################################
# Script to startup and shutdown Oracle and listener
# Author: neo - http://netkiller.8800.org
# File:/etc/rc.d/init.d/oracle
# chmod 750 /etc/init.d/oracle
# chkconfig --add oracle --level 0356
##############################################################
# Setup environment for script execution
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/10.2.0.1/
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/Apache/Apache/bin:$PATH
export NLS_LANG='croatian_croatia.ee8iso8859p2'
export ORACLE_SID=orcl
export DISPLAY=:0
export USER=oracle
if [ -f ./home/oracle/.bash_profile ]; then
        ./home/oracle/.bash_profile
fi

# Determine and execute action based on command line parameter

# check Oracle db status
function chkdb_status() {

        # set username
        SUSER="scott"
        # set password
        SPASS="123456"

        sqlplus -s /nolog > /dev/null 2>&1 <<EOF
whenever sqlerror exit failure
connect $SUSER/$SPASS
exit success
EOF

        if [ $? -ne 0 ]; then
                echo "Connection failed : DB is down"
                exit 1
        else
                echo "Connection succeeded : DB is up"
fi
}

function isql {
        case "$1" in
                start)
                        echo  "*** Starting Oracle iSQL Plus *** "
                        su - $USER -c "$ORACLE_HOME/bin/isqlplusctl start"
                        echo "*** Note: You can access service at url:  http://$(hostname):5560/isqlplus"
                        ;;
                stop)
                        echo  "*** Stopping Oracle iSQL Plus *** "
                        su - $USER -c "$ORACLE_HOME/bin/isqlplusctl stop"
                        ;;
                *)
                        echo "Usage: $1 isql {start|stop}"
                        ;;
        esac

}

function sqlplus {
        case "$1" in
                start)

su - "$oracle_user"<<EOO
    lsnrctl start
    apachectl start
    sqlplus /nolog<<EOS
      connect / as sysdba
      startup
EOS
EOO
                        ;;
                stop)
su - "$oracle_user"<<EOO
    lsnrctl stop
    apachectl stop
    sqlplus /nolog<<EOS
      connect / as sysdba
      shutdown immediate
EOS
EOO
                        ;;
                *)
                        echo "Usage: $1 emctl {start|stop}"
                        ;;
        esac

}
function emctl {
        case "$1" in
                start)
                        echo  "*** Starting Oracle Enterprise Manager 10g Database Control ***"
                        su - $USER -c "$ORACLE_HOME/bin/emctl start dbconsole"
                        echo "*** Note: You can access service at url:  http://$(hostname):1158/em"
                        ;;
                stop)
                        echo  "*** Stopping Oracle Enterprise Manager 10g Database Control ***"
                        su - $USER -c "$ORACLE_HOME/bin/emctl stop dbconsole"
                        ;;
                *)
                        echo "Usage: $1 emctl {start|stop}"
                        ;;
        esac
}
case "$1" in
        start)
                echo "Starting Oracle database(s) listed in /etc/oratab ..."
                sleep 2
                su - $USER -c "$ORACLE_HOME/bin/dbstart"
                echo "Starting TNS listener ..."
                sleep 2
                su - $USER -c "$ORACLE_HOME/bin/lsnrctl start"
                touch /var/lock/subsys/orcl
                ;;
        stop)
                echo "Shutting down TNS listener ..."
                sleep 2
                su - $USER -c "$ORACLE_HOME/bin/lsnrctl stop"
                echo "Shutting down Oracle database(s) listed in /etc/oratab ..."
                su - $USER -c "$ORACLE_HOME/bin/dbshut"
                rm -f /var/lock/subsys/orcl
                ;;
        status)
                chkdb_status
                ps -ax | grep -e ora_ -e tnslsnr
                ;;
        isql)
                isql $2
                ;;
        sqlplus)
                sqlplus $2
                ;;
        emctl)
                emctl $2
                ;;
        *)
                echo "Usage: $1 {start|stop|status}"
                echo
                echo "Usage: $1 [isql | sqlplus | emctl] {start|stop}"
                ;;
esac
exit 0


第 33 章 Oracle Gui

目录

33.1. Create instance
33.2. Oracle Net Configuration Assistant
33.3. Oracle Enterprise Manager
33.4. Other GUI - phpOraAdmin

33.1. Create instance

[oracle@wcs ~]$ dbca


33.2. Oracle Net Configuration Assistant


33.3. Oracle Enterprise Manager

启动em

emctl start dbconsole
[oracle@oracle ~]$ emctl start dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.1.0
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
https://oracle.example.com:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 11g Database Control ..... started.
------------------------------------------------------------------
Logs are generated in directory /opt/oracle/product/11.2.0/dbhome_1/oracle.example.com_wcsdb/sysman/log

https://oracle.example.com:1158/em

使用system用户登录

停止em

emctl stop dbconsole
[oracle@oracle ~]$ emctl stop dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.1.0
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
https://oracle.example.com:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.


33.4. Other GUI - phpOraAdmin

http://phporaadmin.sourceforge.net


第 34 章 CLI

目录

34.1. SQL*Plus
34.1.1. startup/shutdown
34.1.2. conn
34.1.3. parameter
34.1.3.1. db
34.1.3.2. instance_name
34.1.3.3. service_name
34.1.3.4. global_name
34.1.3.5. db_name
34.1.3.6. db_domain
34.1.3.7. sga
34.1.3.8. size
34.1.3.9. spfile
34.1.3.10. cache
34.1.4. $ORACLE_HOME/sqlplus/admin/glogin.sql
34.1.5. @运行SQL
34.2. lsnrctl
34.3. RMAN
34.3.1. 数据库模式
34.3.2. 完全备份
34.3.3. 增量备份
34.3.4. 恢复数据库
34.3.5. 是用tar打包rman文件

34.1. SQL*Plus

[oracle@wcs ~]$ sqlplus /nolog

SQL*Plus: Release 11.2.0.1.0 Production on Sat May 28 18:19:53 2011

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

SQL> conn / as sysdba;
Connected to an idle instance.
SQL> exit

[oracle@wcs ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Sat May 28 18:31:25 2011

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL>

34.1.1. startup/shutdown

[oracle@localhost ~]$ sqlplus

SQL*Plus: Release 10.2.0.1.0 - Production on Tue Jan 5 09:44:13 2010

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

Enter user-name: sys as sysdba
Enter password:

SYS@orcl> startup

SYS@orcl> shutdown immediate

34.1.2. conn

SQL> conn / as sysdba;

34.1.3. parameter

34.1.3.1. db

SQL> show parameter db;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_16k_cache_size                    big integer 0
db_2k_cache_size                     big integer 0
db_32k_cache_size                    big integer 0
db_4k_cache_size                     big integer 0
db_8k_cache_size                     big integer 0
db_block_buffers                     integer     0
db_block_checking                    string      FALSE
db_block_checksum                    string      TYPICAL
db_block_size                        integer     8192
db_cache_advice                      string      ON
db_cache_size                        big integer 0

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest                  string
db_create_online_log_dest_1          string
db_create_online_log_dest_2          string
db_create_online_log_dest_3          string
db_create_online_log_dest_4          string
db_create_online_log_dest_5          string
db_domain                            string      example.com
db_file_multiblock_read_count        integer     128
db_file_name_convert                 string
db_files                             integer     200
db_flash_cache_file                  string

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_flash_cache_size                  big integer 0
db_flashback_retention_target        integer     1440
db_keep_cache_size                   big integer 0
db_lost_write_protect                string      NONE
db_name                              string      orcl
db_recovery_file_dest                string      /opt/oracle/flash_recovery_are
                                                 a
db_recovery_file_dest_size           big integer 3882M
db_recycle_cache_size                big integer 0
db_securefile                        string      PERMITTED
db_ultra_safe                        string      OFF

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_unique_name                       string      orcl
db_writer_processes                  integer     1
dbwr_io_slaves                       integer     0
rdbms_server_dn                      string
standby_archive_dest                 string      ?/dbs/arch
standby_file_management              string      MANUAL
xml_db_events                        string      enable

34.1.3.2. instance_name

SQL> show parameter instance_name;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
instance_name                        string      orcl
SQL>

SQL> select instance from v$thread;

INSTANCE
--------------------------------------------------------------------------------
orcl

34.1.3.3. service_name

SQL> show parameter service_name;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
service_names                        string      orcl.example.com
SQL>

34.1.3.4. global_name

SQL> select * from global_name;

GLOBAL_NAME
--------------------------------------------------------------------------------
ORCL.EXAMPLE.COM

34.1.3.5. db_name

SQL> show parameter db_name;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_name                              string      orcl

SQL> select name from v$database;

NAME
---------
ORCL

34.1.3.6. db_domain

SQL> show parameter db_domain;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_domain                            string      example.com
SQL>

34.1.3.7. sga

SQL> show parameter sga;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
lock_sga                             boolean     FALSE
pre_page_sga                         boolean     FALSE
sga_max_size                         big integer 6016M
sga_target                           big integer 0

34.1.3.8. size

SQL> show parameter size

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
bitmap_merge_area_size               integer     1048576
client_result_cache_size             big integer 0
create_bitmap_area_size              integer     8388608
db_16k_cache_size                    big integer 0
db_2k_cache_size                     big integer 0
db_32k_cache_size                    big integer 0
db_4k_cache_size                     big integer 0
db_8k_cache_size                     big integer 0
db_block_size                        integer     8192
db_cache_size                        big integer 0
db_flash_cache_size                  big integer 0

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_keep_cache_size                   big integer 0
db_recovery_file_dest_size           big integer 3882M
db_recycle_cache_size                big integer 0
global_context_pool_size             string
hash_area_size                       integer     131072
java_max_sessionspace_size           integer     0
java_pool_size                       big integer 0
large_pool_size                      big integer 0
max_dump_file_size                   string      unlimited
object_cache_max_size_percent        integer     10
object_cache_optimal_size            integer     102400

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
olap_page_pool_size                  big integer 0
parallel_execution_message_size      integer     16384
result_cache_max_size                big integer 16064K
sga_max_size                         big integer 6272M
shared_pool_reserved_size            big integer 36909875
shared_pool_size                     big integer 0
sort_area_retained_size              integer     0
sort_area_size                       integer     65536
streams_pool_size                    big integer 0
workarea_size_policy                 string      AUTO

34.1.3.9. spfile

SQL> show parameter spfile ;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      /opt/oracle/product/11.2.0/dbh
                                                 ome_1/dbs/spfilewcsdb.ora

34.1.3.10. cache

SQL> show parameter cache

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
client_result_cache_lag              big integer 3000
client_result_cache_size             big integer 0
db_16k_cache_size                    big integer 0
db_2k_cache_size                     big integer 0
db_32k_cache_size                    big integer 0
db_4k_cache_size                     big integer 0
db_8k_cache_size                     big integer 0
db_cache_advice                      string      ON
db_cache_size                        big integer 0
db_flash_cache_file                  string
db_flash_cache_size                  big integer 0

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_keep_cache_size                   big integer 0
db_recycle_cache_size                big integer 0
object_cache_max_size_percent        integer     10
object_cache_optimal_size            integer     102400
result_cache_max_result              integer     5
result_cache_max_size                big integer 16064K
result_cache_mode                    string      MANUAL
result_cache_remote_expiration       integer     0
session_cached_cursors               integer     50

34.1.4. $ORACLE_HOME/sqlplus/admin/glogin.sql

set line 2000
set linesize 2000
set pagesize 100
col ename format a30
col sal format 999,999.999

34.1.5. @运行SQL

SQL> @ /home/oracle/your.sql

set pagesize 0
set linesize 80
set term off
set feed off
set echo off
set show off
set veri off
set head off

spool outputfile
select * from dba_users;
/
spool off


34.2. lsnrctl


34.3. RMAN

34.3.1. 数据库模式

数据库必须是归档模式，使用下面SQL查询当前数据库模式

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     66
Next log sequence to archive   68
Current log sequence           68

如果已经是归档模式可跳过此步,下面是切换服务器到归档模式的方法：

[oracle@oracle ~]$ sqlplus /nolog (启动sqlplus)

SQL> conn / as sysdba (以DBA身份连接数据库)

SQL> shutdown immediate; (立即关闭数据库)

SQL> startup mount (启动实例并加载数据库，但不打开)

SQL> alter database archivelog; (更改数据库为归档模式)

SQL> alter database open; (打开数据库)

SQL> alter system archive log start; (启用自动归档)

SQL> exit (退出)

rman 采用块备份，查看块信息使用下面SQL语句

select * from dba_extents

34.3.2. 完全备份

run {
allocate channel d1 type disk;
backup format='/opt/oracle/backup/%d_%N_%s.bk' tablespace users;
release channel d1;
}

下面让我来演示给你看

[oracle@oracle ~]$ mkdir /opt/oracle/backup
[oracle@oracle ~]$ rman target sys/passw0rd nocatalog

Recovery Manager: Release 11.2.0.1.0 - Production on Mon Jun 13 12:03:20 2011

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

connected to target database: WCSDB (DBID=2970836713)
using target database control file instead of recovery catalog

RMAN> run {
allocate channel d1 type disk;
backup format='/opt/oracle/backup/%d_%N_%s.bk' tablespace users;
release channel d1;
} 2> 3> 4> 5>

allocated channel: d1
channel d1: SID=36 device type=DISK

Starting backup at 13-JUN-11
channel d1: starting full datafile backup set
channel d1: specifying datafile(s) in backup set
input datafile file number=00004 name=/opt/oracle/oradata/wcsdb/users01.dbf
channel d1: starting piece 1 at 13-JUN-11
channel d1: finished piece 1 at 13-JUN-11
piece handle=/opt/oracle/backup/WCSDB_USERS_1.bk tag=TAG20110613T120325 comment=NONE
channel d1: backup set complete, elapsed time: 00:00:01
Finished backup at 13-JUN-11

released channel: d1

RMAN>

查看备份结果

RMAN> list backup of tablespace users;

List of Backup Sets
===================

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
1       Full    1.30M      DISK        00:00:01     13-JUN-11
        BP Key: 1   Status: AVAILABLE  Compressed: NO  Tag: TAG20110613T120325
        Piece Name: /opt/oracle/backup/WCSDB_USERS_1.bk
  List of Datafiles in backup set 1
  File LV Type Ckp SCN    Ckp Time  Name
  ---- -- ---- ---------- --------- ----
  4       Full 1561686    13-JUN-11 /opt/oracle/oradata/wcsdb/users01.dbf

备份目录下面是刚刚生成的备份文件

$ ls /opt/oracle/backup
WCSDB_USERS_1.bk

例 34.1. full backup

# crontab -u oracle -l
0 1 * * * /opt/oracle/rman/backup.sh

# cat .bash_profile
export ORACLE_BASE=/opt/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
export ORACLE_SID=orcl
export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib

# cat /opt/oracle/rman/backup.sh
rman target sys/passw0rd@orcl msglog=/opt/oracle/rman/log/`date +"%Y%m%d%H%M"`.log cmdfile=/opt/oracle/rman/orcl.rman

# cat /opt/oracle/rman/orcl.rman
run {
   allocate channel c1 type disk;
   allocate channel c2 type disk;
   configure retention policy to recovery window of 6 days;
   configure controlfile autobackup on;
   configure backup optimization  on;
   configure device type disk parallelism 4 backup type to compressed backupset;
   configure controlfile autobackup format for device type disk to '/opt/oracle/backup/rman/%F.ctl';
   sql 'alter system switch logfile';
   backup full database format '/opt/oracle/backup/rman/df_%t_%s_%p.bak' tag='full' include current controlfile;
   sql 'alter system archive log current';
   backup  archivelog all  format '/opt/oracle/backup/rman/arc_%U_%s.bak' delete all input;
   release channel c1;
   release channel c2;
}
crosscheck backup;
delete noprompt expired backup;
delete noprompt obsolete;
exit

34.3.3. 增量备份

RMAN> BACKUP INCREMENTAL LEVEL 0 DATABASE;
RMAN> BACKUP INCREMENTAL LEVEL 1 TABLESPACE SYSTEM DATAFILE 'ora_home/oradata/ tools01.dbf';
RMAN> BACKUP INCREMENTAL LEVEL = 1 CUMULATIVE TABLESPACE users;

网上发现的脚步，署名不详

run{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate channel c3 type disk;
backup full tag 'dbfull' format '/u01/oradata/backup/full%u_%s_%p' database include current controlfile;
sql 'alter system archive log current';
backup filesperset 3 format '/u01/oradata/backup/arch%u_%s_%p' archivelog all delete input;
release channel c1;
release channel c2;
release channel c3;
}

零级备份脚本
run{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate channel c3 type disk;
backup incremental level 0 tag ‘db0’ format ‘/u01/oradata/backup/db0%u_%s_%p’ database skip readonly;
sql ‘alter system archive log current’;
backup filesperset 3 format ‘/u01/oradata/backup/arch%u_%s_%p’ archivelog all delete input;
release channel c1;
release channel c2;
release channel c3;
}

一级备份脚本
run{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate channel c3 type disk;
backup incremental level 1 tag ‘db1’ format ‘/u01/oradata/backup/db1%u_%s_%p’ database skip readonly;
sql ‘alter system archive log current’;
backup filesperset 3 format ‘/u01/oradata/backup/arch%u_%s_%p’ archivelog all delete input;
release channel c1;
release channel c2;
release channel c3;
}

网上发现的脚步，没有署名，我也没有测试过，仅供参考

do_rman.sh

#!/bin/bash
#set env
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export PATH=$ORACLE_HOME/bin:$PATH

TARGET_SID=$TARGET_SID
RMAN_SID=$RMAN_SID

export PATH=$ORACLE_HOME/bin:$PATH
DATE=`date +%w`
DATE_2=`date +%Y%m%d`
BACKUP_PATH=$ORACLE_BASE/admin/$ORACLE_SID/rman/backup
LEVEL=$@
BIN=$ORACLE_HOME/bin
# Delete the data backuped last time

rm -rf $BACKUP_PATH/data/$DATE/*

if [ $# != 1 ]; then
    echo "usage: do_rman.sh n
    where n is the rman backup level(0,1,2 is permitted)."
    exit 1
fi
if [ $@ -ne 0 -a $@ -ne 1 -a $@ -ne 2 ]; then
    echo "usage: do_rman.sh n
    where n is the rman backup level(Only 0,1,2 is permitted)."
    exit 2
fi
echo "[do_rman] rman is starting."
if [ $LEVEL = 0 ]; then
    $BIN/rman log $BACKUP_PATH/log/level.$TARGET_SID.$LEVEL.$DATE_2.log < connect target /;
    connect catalog rman/rman@$RMAN_SID;
    resync catalog;
    run{
        allocate channel c1 type disk ;
        crosscheck backupset of archivelog all ;
        backup filesperset 3 format '$BACKUP_PATH/data/$DATE/arch.%d.live.$LEVEL.%t'(archivelog from time 'sysdate-7' all delete input) ;
        delete noprompt expired backupset of archivelog all ;
        release channel c1 ;
    }

    run{
        allocate channel c2 type disk ;
        crosscheck backupset of database ;
        backup incremental level $LEVEL filesperset 3 format '$BACKUP_PATH/data/$DATE/data.%d.live.$LEVEL.%t'(database include current controlfile) ;
        delete noprompt expired backupset of database ;
        delete noprompt obsolete ;
        release channel c2 ;
    }
    exit;
EOF

else
    $BIN/rman log $BACKUP_PATH/log/level.$TARGET_SID.$LEVEL.$DATE_2.log < connect target sys/sys202;
    connect catalog rman/rman@$RMAN_SID;
    resync catalog;
    run{
        allocate channel c1 type disk ;
        crosscheck backupset of archivelog all ;
        backup filesperset 3 format '$BACKUP_PATH/data/$DATE/arch.%d.live.$LEVEL.%t' (archivelog from time 'sysdate-1' all) ;
        delete noprompt expired backupset of archivelog all ;
        release channel c1 ;
    }

    run{
        allocate channel c2 type disk ;
        crosscheck backupset of database ;
        backup incremental level $LEVEL filesperset 3 format '$BACKUP_PATH/data/$DATE/data.%d.live.$LEVEL.%t' (database include current controlfile) ;
        delete noprompt expired backupset of database ;
        delete noprompt obsolete ;
        release channel c2 ;
    }

    exit;
EOF
fi

echo "[do_rman] rman is success."

34.3.4. 恢复数据库

%rman target=rman/rman@mydb

RMAN> startup nomount

RMAN> restore database;

RMAN> recover database;

RMAN> alter database open;

网上找到的文档，署名不详，我没有测试过是否可以运行

设定参数:
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '/arch/rman/controlfile%F.ctnl';
CONFIGURE CHANNEL 1 DEVICE TYPE DISK FORMAT   '/arch/rman/full%t.bak';

数据库rman 全备
rman>backup database plus archivelog delete input;
备份产生的三个文件
-rw-r----- 1 oracle oinstall   7143424 Jan 28 18:05 controlfilec-2719028776-20100128-01.ctnl
-rw-r----- 1 oracle oinstall  41074688 Jan 28 18:03 full709495428.bak
-rw-r----- 1 oracle oinstall 763379712 Jan 28 18:05 full709495432.bak
-rw-r----- 1 oracle oinstall     17920 Jan 28 18:05 full709495518.bak

rman恢复
------------------
1、启动数据库到 nomount 状态
$sqlplus / as sysdba
SQL> startup nomount

2、spfile 恢复
$rman nocatalog
rman> connect target /
run {
allocate channel c1 DEVICE TYPE DISK format '/arch/rman/controlfile%F.ctnl';
restore spfile to pfile '/arch/pfile.ora' from '/arch/rman/controlfilec-2719028776-20100128-01.ctnl';
release channel c1;
}

3、控制文件恢复
run {
allocate channel c1 DEVICE TYPE DISK format '/arch/rman/controlfile%F.ctnl';
restore controlfile from '/arch/rman/controlfilec-2719028776-20100128-01.ctnl';
release channel c1;
}

4、全库恢复
在恢复控制文件的情况下，可以修改数据到 mount状态，进行全库的恢复
rman> alter database mount;
run {
allocate channel c1 device type disk format '/arch/rman/full%t.bak';
restore database;
release channel c1;
}

5、恢复archivelog
run {
allocate channel c1 device type disk format '/arch/rman/full%t.bak';
restore archivelog all;
}
run {
allocate channel c1 device type disk format '/arch/rman/full%t.bak';
restore archivelog from logseq=72 until logseq=73;
}

6、redolog 恢复
SQL>recover database using backup controlfile until cancel;
SQL>alter database open resetlogs;  //现在有redolog 产生了，还有temp表空间文件也生成了或者分开两步执行

SQL>select * from dual;
全库成功恢复

34.3.5. 是用tar打包rman文件

# find /opt/oracle/rman/ -type f -mtime 1 -printf "%CY-%Cm-%Cd %Cr %s %f\n"

tar --newer="2011-07-04" -zcvf backup.tar.gz /opt/oracle/rman/

find /opt/oracle/rman/ -type f -mtime 1 | xargs tar zcvf oracle_2011-07-04.tgz

rsync -azP `find /opt/oracle/rman/ -type f -mtime 1` test@172.16.0.5:/home/test


部分 V. NoSQL

目录

35. MongoDB
35.1. Quickstart
35.1.1. 二进制tar包安装
35.1.2. Ubuntu MongoDB
35.1.3. CentOS MongoDB
35.1.4. MongoDB + Hadoop
35.2. Security and Authentication
35.2.1. 超级管理员
35.2.2. 数据库用户
35.2.3. 删除用户
35.3. Replication
35.4. Drivers
35.4.1. Using MongoDB in PHP
35.5. mongo 命令行
35.5.1. help
35.5.2. show 查看命令
35.5.2.1. show dbs
35.5.2.2. show collections
35.5.2.3. show users
35.5.2.4. show profile
35.5.3. 切换数据库
35.5.4. 索引管理
35.5.5. 记录管理
35.5.5.1. save
35.5.5.2. insert
35.5.5.3. update
35.5.5.4. remove
35.5.5.5. find
35.5.5.6. count()
35.5.5.7. group()
35.5.6. 管道操作
35.6. Map-Reduce
35.6.1. 使用 Map-Reduce 统计Web 服务器 access.log 日志文件
35.7. Mongo Admin UI
35.7.1. RockMongo
35.7.2. MongoVUE
36. Redis
36.1. Redis 安装
36.1.1. CentOS
36.1.2. Ubuntu
36.1.3. Test Redis
36.2. /etc/redis.conf
36.2.1. 密码认证
36.3. redis-cli - Command-line client to redis-server
36.3.1. info
36.3.2. save/bgsave/lastsave
36.3.3. Pub/Sub 订阅与发布
36.4. phpRedisAdmin
37. Cassandra
37.1. Getting Started
37.1.1. Downloading and Installation
37.1.2. Running Cassandra
37.1.3. cli tool
37.1.4. Testing Cassandra
37.2. Configure Cassandra
37.2.1. Envionment variables
37.2.2. log4j.properties
37.2.3. storage-conf.xml
37.3. Keyspace
37.3.1. Schema
37.3.1.1. Keyspace
37.3.1.2. Column family
37.3.2. Keyspace example
37.4. Cluster
37.4.1. Running a cluster
37.4.2. Running a single node
37.4.3. nodetool
38. Hypertable
38.1. Hypertable 安装
38.1.1. Hypertable standalone 单机安装
38.1.2. Hypertable on HDFS(hadoop) 安装
38.1.3. MapR
38.1.4. Ceph
38.1.5. 检验安装
38.2. Code examples
38.2.1. PHP
38.3. HQL
38.3.1. namespace 命名空间管理
38.3.2. Table 表
38.4. FAQ
38.4.1. 切换 DFS Broker
39. Membase
40. TokyoCabinet/Tyrant
41. Flare
42. Voldemort
43. LevelDB
44. CouchDB
45. RethinkDB
46. HyperDex

http://nosql-database.org/


第 35 章 MongoDB

目录

35.1. Quickstart
35.1.1. 二进制tar包安装
35.1.2. Ubuntu MongoDB
35.1.3. CentOS MongoDB
35.1.4. MongoDB + Hadoop
35.2. Security and Authentication
35.2.1. 超级管理员
35.2.2. 数据库用户
35.2.3. 删除用户
35.3. Replication
35.4. Drivers
35.4.1. Using MongoDB in PHP
35.5. mongo 命令行
35.5.1. help
35.5.2. show 查看命令
35.5.2.1. show dbs
35.5.2.2. show collections
35.5.2.3. show users
35.5.2.4. show profile
35.5.3. 切换数据库
35.5.4. 索引管理
35.5.5. 记录管理
35.5.5.1. save
35.5.5.2. insert
35.5.5.3. update
35.5.5.4. remove
35.5.5.5. find
35.5.5.6. count()
35.5.5.7. group()
35.5.6. 管道操作
35.6. Map-Reduce
35.6.1. 使用 Map-Reduce 统计Web 服务器 access.log 日志文件
35.7. Mongo Admin UI
35.7.1. RockMongo
35.7.2. MongoVUE

http://www.mongodb.org/

35.1. Quickstart

35.1.1. 二进制tar包安装

Install MongoDB

wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-1.5.5.tgz

debian:/srv# tar zxf mongodb-linux-x86_64-1.5.5.tgz
debian:/srv# ln -s mongodb-linux-x86_64-1.5.5 mongodb

Create a data directory

By default MongoDB will store data in /data/db, but it won't automatically create that directory. To create it, do:

$ sudo mkdir -p /data/db/
$ sudo chown `id -u` /data/db

You can also tell MongoDB to use a different data directory, with the --dbpath option.

Run and connect to the server

First, start the MongoDB server in one terminal:

$ ./mongodb/bin/mongod

In a separate terminal, start the shell, which will connect to localhost by default:

$ ./mongodb/bin/mongo
> db.foo.save( { a : 1 } )
> db.foo.find()

Congratulations, you've just saved and retrieved your first document with MongoDB!

例 35.1. MongoDB Test

debian:/srv/mongodb/bin# ./mongo
MongoDB shell version: 1.5.5
connecting to: test
[initandlisten] Thu Jul 22 16:42:07 connection accepted from 127.0.0.1:42876 #1
> db.foo.save({a:1})
Thu Jul 22 16:42:23 allocating new datafile /data/db/test.ns, filling with zeroes...
Thu Jul 22 16:42:23 done allocating datafile /data/db/test.ns, size: 16MB,  took 0.025 secs
Thu Jul 22 16:42:23 allocating new datafile /data/db/test.0, filling with zeroes...
Thu Jul 22 16:42:23 done allocating datafile /data/db/test.0, size: 64MB,  took 0.105 secs
[conn1] Thu Jul 22 16:42:23 building new index on { _id: 1 } for test.foo
[conn1] Thu Jul 22 16:42:23 Buildindex test.foo idxNo:0 { name: "_id_", ns: "test.foo", key: { _id: 1 } }
[conn1] Thu Jul 22 16:42:23 done for 0 records 0secs
[conn1] Thu Jul 22 16:42:23 insert test.foo 136ms
> Thu Jul 22 16:42:23 allocating new datafile /data/db/test.1, filling with zeroes...
Thu Jul 22 16:42:24 done allocating datafile /data/db/test.1, size: 128MB,  took 0.228 secs
> db.foo.find()
{ "_id" : ObjectId("4c48046f74050cbf6c9a0ef6"), "a" : 1 }

> use neo
switched to db neo
> db.foo.save({a:1})
Thu Jul 22 16:54:50 allocating new datafile /data/db/neo.ns, filling with zeroes...
Thu Jul 22 16:54:50 done allocating datafile /data/db/neo.ns, size: 16MB,  took 0.026 secs
Thu Jul 22 16:54:50 allocating new datafile /data/db/neo.0, filling with zeroes...
Thu Jul 22 16:54:50 done allocating datafile /data/db/neo.0, size: 64MB,  took 0.122 secs
[conn1] Thu Jul 22 16:54:50 building new index on { _id: 1 } for neo.foo
[conn1] Thu Jul 22 16:54:50 Buildindex neo.foo idxNo:0 { name: "_id_", ns: "neo.foo", key: { _id: 1 } }
Thu Jul 22 16:54:50 allocating new datafile /data/db/neo.1, filling with zeroes...
[conn1] Thu Jul 22 16:54:50 done for 0 records 0.01secs
[conn1] Thu Jul 22 16:54:50 insert neo.foo 164ms
> Thu Jul 22 16:54:50 done allocating datafile /data/db/neo.1, size: 128MB,  took 0.217 secs

> db.foo.find()
{ "_id" : ObjectId("4c48075a74050cbf6c9a0ef7"), "a" : 1 }
>

> db.neo.save({a:1})
[conn1] Thu Jul 22 16:58:32 building new index on { _id: 1 } for neo.neo
[conn1] Thu Jul 22 16:58:32 Buildindex neo.neo idxNo:0 { name: "_id_", ns: "neo.neo", key: { _id: 1 } }
[conn1] Thu Jul 22 16:58:32 done for 0 records 0.024secs
> db.neo.find()
{ "_id" : ObjectId("4c48083874050cbf6c9a0ef8"), "a" : 1 }

Starting Mongo

Running as a Daemon

 $ ./mongod --fork --logpath /var/log/mongodb.log --logappend

35.1.2. Ubuntu MongoDB

$ sudo apt-get install mongodb-server mongodb-clients

$ /etc/init.d/mongodb
Usage: /etc/init.d/mongodb {start|stop|force-stop|restart|force-reload|status}

35.1.3. CentOS MongoDB

# yum install mongodb-server

# chkconfig mongod on

# service mongod start

单独安装客户端

# yum install mongodb

35.1.4. MongoDB + Hadoop

Hadoop Connector

http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-hadoop/

git clone https://github.com/mongodb/mongo-hadoop.git
git checkout release-1.0

# vim build.sbt
hadoopRelease in ThisBuild := "cdh4"

./sbt package

wget --no-check-certificate https://github.com/downloads/mongodb/mongo-java-driver/mongo-2.7.3.jar
cp mongo-2.7.3.jar /usr/lib/hadoop/lib/
cp core/target/mongo-hadoop-core_cdh3u3-1.0.0.jar /usr/lib/hadoop/lib/

待续......


35.2. Security and Authentication

开启认证

# vim /etc/mongodb.conf
auth = true

重载配置文件

# /etc/init.d/mongod reload
Stopping mongod:                                           [  OK  ]
Starting mongod:                                           [  OK  ]

35.2.1. 超级管理员

> use admin
switched to db admin
> db.addUser('neo','chen')
{
        "user" : "neo",
        "readOnly" : false,
        "pwd" : "68ace374737253d87e0ec91d4fcb673d"
}

> db.system.users.find()
{ "_id" : ObjectId("4c481404b9db6474d2fcb76f"), "user" : "neo", "readOnly" : false, "pwd" : "68ace374737253d87e0ec91d4fcb673d" }

> db.auth('neo','chen')
1

35.2.2. 数据库用户

注意，只有创建了超级管理后，下面的操作才会生效

> use neo
switched to db neo
> db.addUser('neo','chen')
{
        "user" : "neo",
        "readOnly" : false,
        "pwd" : "68ace374737253d87e0ec91d4fcb673d"
}

> db.system.users.find()
{ "_id" : ObjectId("4c481404b9db6474d2fcb76f"), "user" : "neo", "readOnly" : false, "pwd" : "68ace374737253d87e0ec91d4fcb673d" }

> db.auth('neo','chen')
1

35.2.3. 删除用户

Deleting Users 删除用户

To delete a user:

db.system.users.remove( { user: username } )


35.3. Replication

创建主：

Mongod –port 28018 –dbpath /var/mongodb/master –master

 创建从：

Mongod –port 28019 –dbpath /var/mongodb/slave –slave –source localhost:28018


35.4. Drivers

35.4.1. Using MongoDB in PHP

Installing the PHP Driver

sudo pecl install mongo

Open your php.ini file and add to it:

extension=mongo.so

例 35.2. Using MongoDB in PHP

[root@subversion html]# cat mongo.php
<?php

// connect
$m = new Mongo('192.168.3.9');

// select a database
$db = $m->comedy;
$collection = $db->cartoons;

// add an element
$obj = array( "title" => "Calvin and Hobbes", "author" => "Bill Watterson" );
$collection->insert($obj);

// add another element, with a different "shape"
$obj = array( "title" => "XKCD", "online" => true );
$collection->insert($obj);

// find everything in the collection
$cursor = $collection->find();

// iterate through the results
foreach ($cursor as $obj) {
    echo $obj["title"] . "\n";
}

// disconnect
$m->close();

?>

[root@subversion html]# php mongo.php
Calvin and Hobbes
XKCD
[root@subversion html]# php mongo.php
Calvin and Hobbes
XKCD
Calvin and Hobbes
XKCD

> use comedy
switched to db comedy
> db.foo.find()
> db.cartoons.find()
{ "_id" : ObjectId("4c481d2b9503c17611000000"), "title" : "Calvin and Hobbes", "author" : "Bill Watterson" }
{ "_id" : ObjectId("4c481d2b9503c17611010000"), "title" : "XKCD", "online" : true }
{ "_id" : ObjectId("4c481d2f9503c17711000000"), "title" : "Calvin and Hobbes", "author" : "Bill Watterson" }
{ "_id" : ObjectId("4c481d2f9503c17711010000"), "title" : "XKCD", "online" : true }
>


35.5. mongo 命令行

# mongo
MongoDB shell version: 2.2.3
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
    http://docs.mongodb.org/
Questions? Try the support group
    http://groups.google.com/group/mongodb-user
>

35.5.1. help

help

db.help()                    help on DB methods
db.foo.help()                help on collection methods

35.5.2. show 查看命令

35.5.2.1. show dbs

show dbs show database names

> show dbs
local	(empty)
logging	0.203125GB
test	0.203125GB

35.5.2.2. show collections

show collections show collections in current database

> show collections
bios
system.indexes

另一种用法是show tables

> show tables
bios
system.indexes

35.5.2.3. show users

show users show users in current database

35.5.2.4. show profile

show profile show most recent system.profile entries with time >= 1ms

> show profile
db.system.profile is empty
Use db.setProfilingLevel(2) will enable profiling
Use db.system.profile.find() to show raw profile entries

35.5.3. 切换数据库

use <db name>                set curent database to <db name>

> use logging
switched to db logging

35.5.4. 索引管理

增加索引：1(ascending),-1(descending)

增加索引

db.foo.ensureIndex({firstname: 1, lastname: 1}, {unique: true});

索引子对象

db.logging.users.ensureIndex({address.city:1})

查看索引信息

db.logging.getIndexes()
[
    {
        "v" : 1,
        "key" : {
            "_id" : 1
        },
        "ns" : "logging.logging",
        "name" : "_id_"
    }
]

db.logging.users.getIndexKeys()

[ { "_id" : 1 }, { "name" : 1 } ]

根据索引名删除索引

> db.logging.users.dropIndex('name_1')
{ "nIndexesWas" : 2, "ok" : 1 }

> db.logging.users.getIndexKeys()
[ { "_id" : 1 } ]

35.5.5. 记录管理

35.5.5.1. save

存储嵌套的对象

db.foo.save({'name':'neo','address':{'city':'shenzhen','post':518000},'phone':[13113668890,13322993040]})

存储数组对象

db.foo.save({'Uid':'netkiller@msn.com','phone':['13322993040','13113668890']})

35.5.5.2. insert

db.bios.insert(
   {
     _id: 1,
     name: { first: 'John', last: 'Backus' },
     birth: new Date('Dec 03, 1924'),
     death: new Date('Mar 17, 2007'),
     contribs: [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ],
     awards: [
               {
                 award: 'W.W. McDowell Award',
                 year: 1967,
                 by: 'IEEE Computer Society'
               },
               {
                 award: 'National Medal of Science',
                 year: 1975,
                 by: 'National Science Foundation'
               },
               {
                 award: 'Turing Award',
                 year: 1977,
                 by: 'ACM'
               },
               {
                 award: 'Draper Prize',
                 year: 1993,
                 by: 'National Academy of Engineering'
               }
             ]
   }
)

35.5.5.3. update

根据query条件修改，如果不存在则插入，允许修改多条记录

db.foo.update({'yy':5},{'$set':{'xx':2}},upsert=true,multi=true)

35.5.5.4. remove

删除uid=10的记录

db.foo.remove({'uid':10})

删除所有的记录

db.foo.remove()

35.5.5.5. find

查找所有 所有记录

db.foo.find()                list objects in collection foo
db.foo.find( { a : 1 } )     list objects in foo where a == 1

查找一条记录

db.foo.findOne()

根据条件检索10条记录

db.foo.find({'name':'neo'}).limit(10)

sort排序

db.foo.find({'name':'neo'}).sort({'Dt',-1})
db.foo.find().sort({'Ct':-1}).limit(1)

count记录统计操作

db.foo.count()

distinct操作,去重复查询指定列，

db.foo.distinct('name')

”>=”操作

db.foo.find({"timestamp": {"$gte" : 2}})

子对象的查找

db.foo.find({'address.city':'shenzhen'})

35.5.5.6. count()

> db.access.count()
51528
> db.access.count()
104401

35.5.5.7. group()

group()类似SQL中的Group by

> db.test.group({key: {remote_addr: true}, initial: {count: 0}, reduce: function(obj, prev) {prev.count++}});
[
    {
        "remote_addr" : "192.168.2.76",
        "count" : 3
    },
    {
        "remote_addr" : "192.168.2.70",
        "count" : 1
    }
]

35.5.6. 管道操作

cat data.bson | mongo test


35.6. Map-Reduce

35.6.1. 使用 Map-Reduce 统计Web 服务器 access.log 日志文件

首先将web服务器access.log导入到mongodb,参考 http://netkiller.github.io/article/log.html 格式如下：

{
    "_id" : ObjectId("51553efcd8616be7e5395c0d"),
    "remote_addr" : "192.168.2.76",
    "remote_user" : "-",
    "time_local" : "29/Mar/2013:09:20:31 +0800",
    "request" : "GET /tw/ad.jpg HTTP/1.1",
    "status" : "200",
    "body_bytes_sent" : "5557",
    "http_referer" : "http://www.example.com/tw/",
    "http_user_agent" : "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17",
    "http_x_forwarded_for" : "-"
}

创建map方法

var mapFunction1 = function() {
    emit(this.remote_addr, {count:1});
};

创建reduce方法

var reduceFunction1 = function(key, values) {
    var total = 0;
    values.forEach(function (value) {total += value.count;});
    return {ipaddr: key, count:total};
};

分析数据

db.access.mapReduce(mapFunction1, reduceFunction1, {out : "resultCollection"});

输出结果

db.resultCollection.find();

Demo 数据库

> db.resultCollection.find();
{ "_id" : "192.168.2.109", "value" : { "ipaddr" : "192.168.2.109", "count" : 554 } }
{ "_id" : "192.168.2.38", "value" : { "ipaddr" : "192.168.2.38", "count" : 26 } }
{ "_id" : "192.168.2.39", "value" : { "ipaddr" : "192.168.2.39", "count" : 72 } }
{ "_id" : "192.168.2.40", "value" : { "ipaddr" : "192.168.2.40", "count" : 3564 } }
{ "_id" : "192.168.2.49", "value" : { "ipaddr" : "192.168.2.49", "count" : 955 } }
{ "_id" : "192.168.2.5", "value" : { "ipaddr" : "192.168.2.5", "count" : 2 } }
{ "_id" : "192.168.2.76", "value" : { "ipaddr" : "192.168.2.76", "count" : 60537 } }
{ "_id" : "192.168.3.12", "value" : { "ipaddr" : "192.168.3.12", "count" : 9577 } }
{ "_id" : "192.168.3.14", "value" : { "ipaddr" : "192.168.3.14", "count" : 343 } }
{ "_id" : "192.168.3.18", "value" : { "ipaddr" : "192.168.3.18", "count" : 1006 } }
{ "_id" : "192.168.3.26", "value" : { "ipaddr" : "192.168.3.26", "count" : 2714 } }
{ "_id" : "192.168.6.19", "value" : { "ipaddr" : "192.168.6.19", "count" : 668 } }
{ "_id" : "192.168.6.2", "value" : { "ipaddr" : "192.168.6.2", "count" : 123760 } }
{ "_id" : "192.168.6.30", "value" : { "ipaddr" : "192.168.6.30", "count" : 1196 } }
{ "_id" : "192.168.6.35", "value" : { "ipaddr" : "192.168.6.35", "count" : 1050 } }
>


35.7. Mongo Admin UI

http://docs.mongodb.org/ecosystem/tools/administration-interfaces/

35.7.1. RockMongo

http://code.google.com/p/rock-php/

35.7.2. MongoVUE

http://blog.mongovue.com/

MongoVUE 是收费软件，提供的功能比较完善，可以免费试用


第 36 章 Redis

目录

36.1. Redis 安装
36.1.1. CentOS
36.1.2. Ubuntu
36.1.3. Test Redis
36.2. /etc/redis.conf
36.2.1. 密码认证
36.3. redis-cli - Command-line client to redis-server
36.3.1. info
36.3.2. save/bgsave/lastsave
36.3.3. Pub/Sub 订阅与发布
36.4. phpRedisAdmin

http://redis.io/

36.1. Redis 安装

36.1.1. CentOS

安装fedora的YUM源，

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-7.noarch.rpm

安装redis

# yum install redis

# chkconfig redis on

# service redis start

备份配置文件，

# cp /etc/redis.conf /etc/redis.conf.original

36.1.2. Ubuntu

$ sudo apt-get install redis-server

$ dpkg -s redis-server
Package: redis-server
Status: install ok installed
Priority: optional
Section: database
Installed-Size: 208
Maintainer: Chris Lamb <lamby@debian.org>
Architecture: amd64
Source: redis
Version: 2:1.2.6-1
Depends: libc6 (>= 2.7), adduser
Conffiles:
 /etc/redis/redis.conf a19bad63017ec19def2c3a8a07bdc362
 /etc/logrotate.d/redis-server 06755b99ef70d62a56cff94cbfc36de7
 /etc/init.d/redis-server 3742555c10ab16fdd67fcbaf92faf694
 /etc/bash_completion.d/redis-cli 848565df7f222dc03c8d5cb34b9e0188
Description: Persistent key-value database with network interface
 Redis is a key-value database in a similar vein to memcache but the dataset
 is non-volatile. Redis additionally provides native support for atomically
 manipulating and querying data structures such as lists and sets.
 .
 The dataset is stored entirely in memory and periodically flushed to disk.
Homepage: http://code.google.com/p/redis/

$ cat /etc/redis/redis.conf

# Redis configuration file example

# By default Redis does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
daemonize yes

# When run as a daemon, Redis write a pid file in /var/run/redis.pid by default.
# You can specify a custom pid file location here.
pidfile /var/run/redis.pid

# Accept connections on the specified port, default is 6379
port 6379

# If you want you can bind a single interface, if the bind option is not
# specified all the interfaces will listen for connections.
#
bind 127.0.0.1

# Close the connection after a client is idle for N seconds (0 to disable)
timeout 300

# Set server verbosity to 'debug'
# it can be one of:
# debug (a lot of information, useful for development/testing)
# notice (moderately verbose, what you want in production probably)
# warning (only very important / critical messages are logged)
loglevel notice

# Specify the log file name. Also 'stdout' can be used to force
# the demon to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
logfile /var/log/redis/redis-server.log

# Set the number of databases. The default database is DB 0, you can select
# a different one on a per-connection basis using SELECT <dbid> where
# dbid is a number between 0 and 'databases'-1
databases 16

################################ SNAPSHOTTING  #################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
save 900 1
save 300 10
save 60 10000

# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes

# The filename where to dump the DB
dbfilename dump.rdb

# For default save/load DB in/from the working directory
# Note that you must specify a directory not a file name.
dir /var/lib/redis

################################# REPLICATION #################################

# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# another Redis server. Note that the configuration is local to the slave
# so for example it is possible to configure the slave to save the DB with a
# different interval, or to listen to another port, and so on.
#
# slaveof <masterip> <masterport>

# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the slave to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the slave request.
#
# masterauth <master-password>

################################## SECURITY ###################################

# Require clients to issue AUTH <PASSWORD> before processing any other
# commands.  This might be useful in environments in which you do not trust
# others with access to the host running redis-server.
#
# This should stay commented out for backward compatibility and because most
# people do not need auth (e.g. they run their own servers).
#
# requirepass foobared

################################### LIMITS ####################################

# Set the max number of connected clients at the same time. By default there
# is no limit, and it's up to the number of file descriptors the Redis process
# is able to open. The special value '0' means no limts.
# Once the limit is reached Redis will close all the new connections sending
# an error 'max number of clients reached'.
#
# maxclients 128

# Don't use more memory than the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys with an
# EXPIRE set. It will try to start freeing keys that are going to expire
# in little time and preserve keys with a longer time to live.
# Redis will also try to remove objects from free lists if possible.
#
# If all this fails, Redis will start to reply with errors to commands
# that will use more memory, like SET, LPUSH, and so on, and will continue
# to reply to most read-only commands like GET.
#
# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a
# 'state' server or cache, not as a real DB. When Redis is used as a real
# database the memory usage will grow over the weeks, it will be obvious if
# it is going to use too much memory in the long run, and you'll have the time
# to upgrade. With maxmemory after the limit is reached you'll start to get
# errors for write operations, and this may even lead to DB inconsistency.
#
# maxmemory <bytes>

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. If you can live
# with the idea that the latest records will be lost if something like a crash
# happens this is the preferred way to run Redis. If instead you care a lot
# about your data and don't want to that a single record can get lost you should
# enable the append only mode: when this mode is enabled Redis will append
# every write operation received in the file appendonly.log. This file will
# be read on startup in order to rebuild the full dataset in memory.
#
# Note that you can have both the async dumps and the append only file if you
# like (you have to comment the "save" statements above to disable the dumps).
# Still if append only mode is enabled Redis will load the data from the
# log file at startup ignoring the dump.rdb file.
#
# The name of the append only file is "appendonly.log"
#
# IMPORTANT: Check the BGREWRITEAOF to check how to rewrite the append
# log file in background when it gets too big.

appendonly no

# The fsync() call tells the Operating System to actually write data on disk
# instead to wait for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log . Slow, Safest.
# everysec: fsync only if one second passed since the last fsync. Compromise.
#
# The default is "always" that's the safer of the options. It's up to you to
# understand if you can relax this to "everysec" that will fsync every second
# or to "no" that will let the operating system flush the output buffer when
# it want, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting).

appendfsync always
# appendfsync everysec
# appendfsync no

############################### ADVANCED CONFIG ###############################

# Glue small output buffers together in order to send small replies in a
# single TCP packet. Uses a bit more CPU but most of the times it is a win
# in terms of number of queries per second. Use 'yes' if unsure.
glueoutputbuf yes

# Use object sharing. Can save a lot of memory if you have many common
# string in your dataset, but performs lookups against the shared objects
# pool so it uses more CPU and can be a bit slower. Usually it's a good
# idea.
#
# When object sharing is enabled (shareobjects yes) you can use
# shareobjectspoolsize to control the size of the pool used in order to try
# object sharing. A bigger pool size will lead to better sharing capabilities.
# In general you want this value to be at least the double of the number of
# very common strings you have in your dataset.
#
# WARNING: object sharing is experimental, don't enable this feature
# in production before of Redis 1.0-stable. Still please try this feature in
# your development environment so that we can test it better.
shareobjects no
shareobjectspoolsize 1024

$ sudo /etc/init.d/redis-server start

36.1.3. Test Redis

http://redis.io/commands

$ redis-cli info
redis_version:1.2.6
arch_bits:64
multiplexing_api:epoll
uptime_in_seconds:859
uptime_in_days:0
connected_clients:1
connected_slaves:0
used_memory:619490
used_memory_human:604.97K
changes_since_last_save:0
bgsave_in_progress:0
last_save_time:1311100746
bgrewriteaof_in_progress:0
total_connections_received:4
total_commands_processed:0
role:master

$ redis-cli set name neo
OK
$ redis-cli get name
neo

$ telnet localhost 6379
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
get name
$3
neo
quit
Connection closed by foreign host.


36.2. /etc/redis.conf

36.2.1. 密码认证

打开 /etc/redis.conf 修改 requirepass 配置项

# vim /etc/redis.conf

requirepass test123

测试

# service redis restart
Stopping redis-server:                                     [  OK  ]
Starting redis-server:                                     [  OK  ]

# redis-cli
redis 127.0.0.1:6379> set h helloworld
(error) ERR operation not permitted

auth test123

redis 127.0.0.1:6379> auth test123
OK
redis 127.0.0.1:6379> set h helloworld
OK
redis 127.0.0.1:6379> get h
"helloworld"
redis 127.0.0.1:6379> exit

redis-cli -a 参数指定密码

# redis-cli -a test123
redis 127.0.0.1:6379> set h helloworld
OK
redis 127.0.0.1:6379> get h
"helloworld"
redis 127.0.0.1:6379> exit

通过 config 动态改变密码，无需重新启动 redis 进程

# redis-cli -a test123
redis 127.0.0.1:6379> config get requirepass
1) "requirepass"
2) "test123"
redis 127.0.0.1:6379> config set requirepass passabc
OK
redis 127.0.0.1:6379> auth passabc
OK
redis 127.0.0.1:6379> set h helloworld
OK
redis 127.0.0.1:6379> get h
"helloworld"
redis 127.0.0.1:6379> exit

注意：config 不能保存到配置文件

# grep requirepass /etc/redis.conf

# If the master is password protected (using the "requirepass" configuration
# requirepass foobared
requirepass test123

master/slave 模式， master 有密码, slave 怎样配置？

masterauth  password


36.3. redis-cli - Command-line client to redis-server

36.3.1. info

redis 127.0.0.1:6379> info
redis_version:2.4.10
redis_git_sha1:00000000
redis_git_dirty:0
arch_bits:64
multiplexing_api:epoll
gcc_version:4.4.6
process_id:29663
uptime_in_seconds:1189
uptime_in_days:0
lru_clock:1018411
used_cpu_sys:0.10
used_cpu_user:0.09
used_cpu_sys_children:0.01
used_cpu_user_children:0.00
connected_clients:1
connected_slaves:0
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
used_memory:730664
used_memory_human:713.54K
used_memory_rss:7225344
used_memory_peak:730720
used_memory_peak_human:713.59K
mem_fragmentation_ratio:9.89
mem_allocator:jemalloc-2.2.5
loading:0
aof_enabled:0
changes_since_last_save:0
bgsave_in_progress:0
last_save_time:1373332622
bgrewriteaof_in_progress:0
total_connections_received:4
total_commands_processed:14
expired_keys:0
evicted_keys:0
keyspace_hits:3
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:744
vm_enabled:0
role:master
db0:keys=4,expires=0
redis 127.0.0.1:6379>

36.3.2. save/bgsave/lastsave

save/bgsave 保存持久化将数据，lastsave 查看相关信息

redis 127.0.0.1:6379> save
OK
redis 127.0.0.1:6379> bgsave
Background saving started
redis 127.0.0.1:6379> lastsave
(integer) 1373335757

36.3.3. Pub/Sub 订阅与发布

redis 提供基本的MQ 功能，下面我们做一个演示

开启第一个终端窗口，订阅first second

$ redis-cli
redis 127.0.0.1:6379> SUBSCRIBE first second
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "first"
3) (integer) 1
1) "subscribe"
2) "second"
3) (integer) 2

开启第二个终端窗口，分别发送first second

$ redis-cli
redis 127.0.0.1:6379>  PUBLISH second Hello
(integer) 1
redis 127.0.0.1:6379>  PUBLISH first Helloworld!!!
(integer) 1
redis 127.0.0.1:6379> quit

现在切换到第一个终端窗口，应该能够看到发送过来的字符串

$ redis-cli
redis 127.0.0.1:6379> SUBSCRIBE first second
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "first"
3) (integer) 1
1) "subscribe"
2) "second"
3) (integer) 2

1) "message"
2) "second"
3) "Hello"

1) "message"
2) "first"
3) "Helloworld!!!"


36.4. phpRedisAdmin

https://github.com/ErikDubbelboer/phpRedisAdmin

Example

You can find an example database at http://dubbelboer.com/phpRedisAdmin/

第一种方法

git clone https://github.com/ErikDubbelboer/phpRedisAdmin
cd phpRedisAdmin
git clone https://github.com/nrk/predis

第二种方法

git clone https://github.com/ErikDubbelboer/phpRedisAdmin.git
cd phpRedisAdmin
git submodule init
git submodule update


第 37 章 Cassandra

目录

37.1. Getting Started
37.1.1. Downloading and Installation
37.1.2. Running Cassandra
37.1.3. cli tool
37.1.4. Testing Cassandra
37.2. Configure Cassandra
37.2.1. Envionment variables
37.2.2. log4j.properties
37.2.3. storage-conf.xml
37.3. Keyspace
37.3.1. Schema
37.3.1.1. Keyspace
37.3.1.2. Column family
37.3.2. Keyspace example
37.4. Cluster
37.4.1. Running a cluster
37.4.2. Running a single node
37.4.3. nodetool

http://incubator.apache.org/cassandra/

37.1. Getting Started

37.1.1. Downloading and Installation

$ cd /srv/

$ cd /usr/local/src/

$ sudo wget -c http://apache.freelamp.com/cassandra/0.5.1/apache-cassandra-0.5.1-bin.tar.gz
$ sudo tar zxvf apache-cassandra-0.5.1-bin.tar.gz
$ cp -r /usr/local/src/apache-cassandra-0.5.1 /srv/
$ cd /srv/
$ sudo ln -s apache-cassandra-0.5.1 apache-cassandra
$ cd apache-cassandra

37.1.2. Running Cassandra

Running Cassandra

$ bin/cassandra
$ Listening for transport dt_socket at address: 8888
INFO - Saved Token not found. Using 70882909557229809272696372631016976044
INFO - Starting up server gossip

37.1.3. cli tool

cli

$ bin/cassandra-cli

neo@db:/srv/apache-cassandra$ bin/cassandra-cli
Welcome to cassandra CLI.

Type 'help' or '?' for help. Type 'quit' or 'exit' to quit.
cassandra>

cassandra> connect localhost/9160
Connected to localhost/9160

37.1.4. Testing Cassandra

test

cassandra> show keyspaces
Keyspace1
system

insert value

cassandra> set Keyspace1.Standard1['member']['name']='neo'
Value inserted.
cassandra> set Keyspace1.Standard1['member']['age']='27'
Value inserted.
cassandra> set Keyspace1.Standard1['member']['email']='openunix@163.com'
Value inserted.
cassandra>
cassandra> get Keyspace1.Standard1['member']
=> (column=name, value=neo, timestamp=1271070497471)
=> (column=email, value=openunix@163.com, timestamp=1271070498334)
=> (column=age, value=27, timestamp=1271070497519)
Returned 3 results.
cassandra>


37.2. Configure Cassandra

37.2.1. Envionment variables

CASSANDRA_HOME=/srv/apache-cassandra

37.2.2. log4j.properties

[root@db apache-cassandra]# vim conf/log4j.properties
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.file.maxFileSize=20MB
log4j.appender.file.maxBackupIndex=50
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%5p [%t] %d{ISO8601} %F (line %L) %m%n
# Edit the next line to point to your logs directory
log4j.appender.R.File=/var/log/cassandra/system.log

# Application logging options
#log4j.logger.com.facebook=DEBUG
#log4j.logger.com.facebook.infrastructure.gms=DEBUG
#log4j.logger.com.facebook.infrastructure.db=DEBUG

37.2.3. storage-conf.xml

[root@db apache-cassandra]# vim conf/storage-conf.xml


37.3. Keyspace

37.3.1. Schema

37.3.1.1. Keyspace

37.3.1.2. Column family

37.3.1.2.1. Name
37.3.1.2.2. Column
37.3.1.2.3. Super column
37.3.1.2.4. Sorting

37.3.2. Keyspace example

例 37.1. Twitter

<Keyspace Name="Twitter">
<ColumnFamily CompareWith="UTF8Type" Name="Statuses" />
<ColumnFamily CompareWith="UTF8Type" Name="StatusAudits" />
<ColumnFamily CompareWith="UTF8Type" Name="StatusRelationships"
CompareSubcolumnsWith="TimeUUIDType" ColumnType="Super" />
<ColumnFamily CompareWith="UTF8Type" Name="Users" />
<ColumnFamily CompareWith="UTF8Type" Name="UserRelationships"
CompareSubcolumnsWith="TimeUUIDType" ColumnType="Super" />
</Keyspace>

例 37.2. Twissandra

  <Keyspaces>
    <Keyspace Name="Twissandra">
       <ColumnFamily CompareWith="UTF8Type" Name="User"/>
      <ColumnFamily CompareWith="BytesType" Name="Username"/>
      <ColumnFamily CompareWith="BytesType" Name="Friends"/>
      <ColumnFamily CompareWith="BytesType" Name="Followers"/>
      <ColumnFamily CompareWith="UTF8Type" Name="Tweet"/>
      <ColumnFamily CompareWith="LongType" Name="Timeline"/>
      <ColumnFamily CompareWith="LongType" Name="Userline"/>

      <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>

      <!-- Number of replicas of the data -->
      <ReplicationFactor>1</ReplicationFactor>
      <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>

    </Keyspace>
  </Keyspaces>

Schema Layout

In Cassandra, the way that your data is structured is very closely tied to how how it will be retrieved. Let's start with the user ColumnFamily. The key is a user id, and the columns are the properties on the user:

User = {
    'a4a70900-24e1-11df-8924-001ff3591711': {
        'id': 'a4a70900-24e1-11df-8924-001ff3591711',
        'username': 'ericflo',
        'password': '****',
    },
}

Since some of the URLs on the site actually have the username, we need to be able to map from the username to the user id:

Username = {
    'ericflo': {
        'id': 'a4a70900-24e1-11df-8924-001ff3591711',
    },
}

Friends and followers are keyed by the user id, and then the columns are the friend user id and follower user ids, and we store a timestamp as the value because it's interesting information to have:

Friends = {
    'a4a70900-24e1-11df-8924-001ff3591711': {
        # friend id: timestamp of when the friendship was added
        '10cf667c-24e2-11df-8924-001ff3591711': '1267413962580791',
        '343d5db2-24e2-11df-8924-001ff3591711': '1267413990076949',
        '3f22b5f6-24e2-11df-8924-001ff3591711': '1267414008133277',
    },
}

Followers = {
    'a4a70900-24e1-11df-8924-001ff3591711': {
        # friend id: timestamp of when the followership was added
        '10cf667c-24e2-11df-8924-001ff3591711': '1267413962580791',
        '343d5db2-24e2-11df-8924-001ff3591711': '1267413990076949',
        '3f22b5f6-24e2-11df-8924-001ff3591711': '1267414008133277',
    },
}

Tweets are stored in a way similar to users:

Tweet = {
    '7561a442-24e2-11df-8924-001ff3591711': {
        'id': '89da3178-24e2-11df-8924-001ff3591711',
        'user_id': 'a4a70900-24e1-11df-8924-001ff3591711',
        'body': 'Trying out Twissandra. This is awesome!',
        '_ts': '1267414173047880',
    },
}

The Timeline and Userline column families keep track of which tweets should appear, and in what order. To that effect, the key is the user id, the column name is a timestamp, and the column value is the tweet id:

Timeline = {
    'a4a70900-24e1-11df-8924-001ff3591711': {
        # timestamp of tweet: tweet id
        1267414247561777: '7561a442-24e2-11df-8924-001ff3591711',
        1267414277402340: 'f0c8d718-24e2-11df-8924-001ff3591711',
        1267414305866969: 'f9e6d804-24e2-11df-8924-001ff3591711',
        1267414319522925: '02ccb5ec-24e3-11df-8924-001ff3591711',
    },
}

Userline = {
    'a4a70900-24e1-11df-8924-001ff3591711': {
        # timestamp of tweet: tweet id
        1267414247561777: '7561a442-24e2-11df-8924-001ff3591711',
        1267414277402340: 'f0c8d718-24e2-11df-8924-001ff3591711',
        1267414305866969: 'f9e6d804-24e2-11df-8924-001ff3591711',
        1267414319522925: '02ccb5ec-24e3-11df-8924-001ff3591711',
    },
}


37.4. Cluster

37.4.1. Running a cluster

<Seed>127.0.0.1</Seed>

改为

<Seed>172.16.0.1</Seed>

  <ListenAddress>localhost</ListenAddress>
改为：
  <ListenAddress>172.16.0.1</ListenAddress>

    <ThriftAddress>localhost</ThriftAddress>
改为：
    <ThriftAddress>0.0.0.0</ThriftAddress>
$ bin/cassandra

37.4.2. Running a single node

<Seed>127.0.0.1</Seed>

改为

<Seed>172.16.0.2</Seed>

  <Seeds>
      <Seed>172.16.0.1</Seed>
      <Seed>172.16.0.2</Seed>
      <Seed>172.16.0.3</Seed>
      <Seed>172.16.0.4</Seed>
      <Seed>172.16.0.5</Seed>
  </Seeds>

  <ListenAddress>localhost</ListenAddress>
改为：
  <ListenAddress>172.16.0.2</ListenAddress>

    <ThriftAddress>localhost</ThriftAddress>
改为：
    <ThriftAddress>0.0.0.0</ThriftAddress>
$ bin/cassandra

37.4.3. nodetool

nodeprobe -host 172.16.0.1 ring


第 38 章 Hypertable

目录

38.1. Hypertable 安装
38.1.1. Hypertable standalone 单机安装
38.1.2. Hypertable on HDFS(hadoop) 安装
38.1.3. MapR
38.1.4. Ceph
38.1.5. 检验安装
38.2. Code examples
38.2.1. PHP
38.3. HQL
38.3.1. namespace 命名空间管理
38.3.2. Table 表
38.4. FAQ
38.4.1. 切换 DFS Broker

http://hypertable.org/

38.1. Hypertable 安装

Hypertable 的几种安装方式

单机：安装于单机，采用本地文件系统

Hadoop：分布式安装，使用Hadoop(HDFS)作为存储

MapR：分布式安装，在MapR之上

Ceph：分布式安装，在Ceph之上

38.1.1. Hypertable standalone 单机安装

过程 38.1. Hypertable standalone 安装过程

  1. 安装 Hypertable 软件包

    # cd /usr/local/src/
# wget http://cdn.hypertable.com/packages/0.9.7.0/hypertable-0.9.7.0-linux-x86_64.rpm

  2. 安装 Hypertable， 我个人比较喜欢用yum localinstall他会解决软件之间的依赖关系

    # yum localinstall hypertable-0.9.7.0-linux-x86_64.rpm

    相关的软件会自动安装

    Dependencies Resolved

====================================================================================================================
 Package                       Arch          Version                  Repository                               Size
====================================================================================================================
Installing:
 hypertable                    x86_64        0.9.7.0-1                /hypertable-0.9.7.0-linux-x86_64        288 M
Installing for dependencies:
 mailcap                       noarch        2.1.31-2.el6             base                                     27 k
 perl-Bit-Vector               x86_64        7.1-2.el6                base                                    169 k
 perl-Carp-Clan                noarch        6.03-2.el6               base                                     25 k
 perl-Compress-Raw-Zlib        x86_64        1:2.020-127.el6          base                                     68 k
 perl-Compress-Zlib            x86_64        2.020-127.el6            base                                     43 k
 perl-HTML-Parser              x86_64        3.64-2.el6               base                                    109 k
 perl-HTML-Tagset              noarch        3.20-4.el6               base                                     17 k
 perl-IO-Compress-Base         x86_64        2.020-127.el6            base                                     67 k
 perl-IO-Compress-Zlib         x86_64        2.020-127.el6            base                                    134 k
 perl-IO-String                noarch        1.08-9.el6               base                                     15 k
 perl-URI                      noarch        1.40-2.el6               base                                    117 k
 perl-libwww-perl              noarch        5.833-2.el6              base                                    387 k

Transaction Summary
====================================================================================================================
Install      13 Package(s)

  3. Hypertable 默认安装在 /opt/hypertable/0.9.7.0

    备份配置文件，

    # cd /opt/hypertable/0.9.7.0/conf
# cp hypertable.cfg hypertable.cfg.original

  4. FHS-IZE 安装

    # bin/fhsize.sh
Setting up /var/opt/hypertable
Setting up /etc/opt/hypertable
fshize /opt/hypertable/0.9.7.0:  success

  5. 设计 "CURRENT" 连接

    # cd /opt/hypertable
# ln -s 0.9.7.0 current

  6. 安装 notification-hook.sh 脚本.

    # cp conf/notification-hook.tmpl conf/notification-hook.sh
# chmod o+x conf/notification-hook.sh

    测试 notification-hook.sh脚本 .

    /opt/hypertable/current/conf/notification-hook.sh "Test Message" "This is a test."

  7. 启动 hypertable

    # /opt/hypertable/current/bin/start-all-servers.sh local
DFS broker: available file descriptors: 1024
Started DFS Broker (local)
Started Hyperspace
Started Hypertable.Master
/proc/sys/vm/swappiness = 60
Started Hypertable.RangeServer
Started ThriftBroker

# /opt/hypertable/current/bin/ht shell

Welcome to the hypertable command interpreter.
For information about Hypertable, visit http://hypertable.com

Type 'help' for a list of commands, or 'help shell' for a
list of shell meta commands.

hypertable>

  8. 测试安装是否有效

# /opt/hypertable/current/bin/ht shell

Welcome to the hypertable command interpreter.
For information about Hypertable, visit http://hypertable.com

Type 'help' for a list of commands, or 'help shell' for a
list of shell meta commands.

hypertable> help

USE ................ Sets the current namespace
CREATE NAMESPACE ... Creates a new namespace
DROP NAMESPACE ..... Removes a namespace
EXISTS TABLE ....... Check if table exists
CREATE TABLE ....... Creates a table
DELETE ............. Deletes all or part of a row from a table
DESCRIBE TABLE ..... Displays a table's schema
DROP TABLE ......... Removes a table
RENAME TABLE ....... Renames a table
DUMP TABLE ......... Create efficient backup file
ALTER TABLE ........ Add/remove column family from existing table
INSERT ............. Inserts data into a table
LOAD DATA INFILE ... Loads data from a TSV input file into a table
SELECT ............. Selects (and display) cells from a table
SHOW CREATE TABLE .. Displays CREATE TABLE command used to create table
SHOW TABLES ........ Displays only the list of tables in the current namespace
GET LISTING ........ Displays the list of tables and namespace in the current namespace

Statements must be terminated with ';'.  For more information on
a specific statement, type 'help <statement>', where <statement> is from
the preceding list.

hypertable>quit

  9. 停止 hypertable

    运行下列命令停止 Hypertable

    $ /opt/hypertable/current/bin/stop-servers.sh

38.1.2. Hypertable on HDFS(hadoop) 安装

Hadoop - HDFS 安装指南

过程 38.2. Hypertable on HDFS

  1. 创建工作目录

    $ hadoop fs -mkdir /hypertable
$ hadoop fs -chmod 777 /hypertable

  2. 安装 Java 运行环境

    yum install java-1.7.0-openjdk
yum localinstall http://ftp.cuhk.edu.hk/pub/packages/apache.org/hadoop/common/hadoop-1.1.2/hadoop-1.1.2-1.x86_64.rpm

  3. 修改 jrun bug

    cp /opt/hypertable/current/bin/jrun /opt/hypertable/current/bin/jrun.old

vim /opt/hypertable/current/bin/jrun
#HT_JAR=`ls -1 /opt/hypertable/doug/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'`
HT_JAR=`ls -1 /opt/hypertable/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'`

     export JAVA_HOME=/usr
 export HADOOP_HOME=/usr
 export HYPERTABLE_HOME=/opt/hypertable/current

  4. hypertable.cfg

    # cat conf/hypertable.cfg
#
# hypertable.cfg
#

# HDFS Broker
#HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf
HdfsBroker.Hadoop.ConfDir=/etc/hadoop

# Ceph Broker
CephBroker.MonAddr=192.168.6.2:6789

# Local Broker
DfsBroker.Local.Root=fs/local

# DFS Broker - for clients
DfsBroker.Port=38030

# Hyperspace
Hyperspace.Replica.Host=localhost
Hyperspace.Replica.Port=38040
Hyperspace.Replica.Dir=hyperspace

# Hypertable.Master
#Hypertable.Master.Host=localhost
Hypertable.Master.Port=38050

# Hypertable.RangeServer
Hypertable.RangeServer.Port=38060

Hyperspace.KeepAlive.Interval=30000
Hyperspace.Lease.Interval=1000000
Hyperspace.GracePeriod=200000

# ThriftBroker
ThriftBroker.Port=38080

    Hadoop 配置文件 /etc/hadoop/core-site.xml

# cat /etc/hadoop/core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
         <name>fs.default.name</name>
         <value>hdfs://namenode.example.com:9000</value>
    </property>
    <property>
         <name>hadoop.tmp.dir</name>
         <value>/var/tmp/hadoop</value>
    </property>
</configuration>

    Hadoop 配置文件 /etc/hadoop/hdfs-site.xml

# cat /etc/hadoop/hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>dfs.name.dir</name>
        <value>/var/hadoop/name1</value>
        <description>  </description>
    </property>
    <property>
        <name>dfs.data.dir</name>
        <value>/var/hadoop/hdfs/data1</value>
        <description> </description>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
</configuration>

  5. 启动 dfsbroker

    # /opt/hypertable/current/bin/set-hadoop-distro.sh cdh4
Hypertable successfully configured for Hadoop cdh4

    # /opt/hypertable/current/bin/start-dfsbroker.sh hadoop
DFS broker: available file descriptors: 1024
Started DFS Broker (hadoop)

    查看启动日志

    # tail -f /opt/hypertable/current/log/DfsBroker.hadoop.log
log4j:WARN No appenders could be found for logger (org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
HdfsBroker.dfs.client.read.shortcircuit=false
HdfsBroker.dfs.replication=2
HdfsBroker.Server.fs.default.name=hdfs://namenode.example.com:9000
Apr 23, 2013 6:43:18 PM org.hypertable.AsyncComm.IOHandler DeliverEvent
INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Connection Established
Apr 23, 2013 6:43:18 PM org.hypertable.DfsBroker.hadoop.ConnectionHandler handle
INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Disconnect - COMM broken connection : Closing all open handles from /192.168.6.25:53556
Closed 0 input streams and 0 output streams for client connection /192.168.6.25:53556

38.1.3. MapR

38.1.4. Ceph

修改 CephBroker.MonAddr 对应的服务器与端口号即可

# cat hypertable.cfg
#
# hypertable.cfg
#

# HDFS Broker
HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf

# Ceph Broker
CephBroker.MonAddr=192.168.6.2:6789

# Local Broker
DfsBroker.Local.Root=fs/local

# DFS Broker - for clients
DfsBroker.Port=38030

# Hyperspace
Hyperspace.Replica.Host=localhost
Hyperspace.Replica.Port=38040
Hyperspace.Replica.Dir=hyperspace

# Hypertable.Master
Hypertable.Master.Port=38050

# Hypertable.RangeServer
Hypertable.RangeServer.Port=38060

Hyperspace.KeepAlive.Interval=30000
Hyperspace.Lease.Interval=1000000
Hyperspace.GracePeriod=200000

# ThriftBroker
ThriftBroker.Port=38080

启动 dfsbroker

# /opt/hypertable/current/bin/start-dfsbroker.sh ceph

38.1.5. 检验安装

创建一个表

# echo "USE '/'; CREATE TABLE foo ( c1, c2 ); GET LISTING;" \
>     | /opt/hypertable/current/bin/ht shell --batch
foo
sys	(namespace)
tmp	(namespace)

插入一些数据

# echo "USE '/'; INSERT INTO foo VALUES('001', 'c1', 'very'), \
>     ('000', 'c1', 'Hypertable'), ('001', 'c2', 'easy'), ('000', 'c2', 'is');" \
>     | /opt/hypertable/current/bin/ht shell --batch

查询数据

# echo "USE '/'; SELECT * FROM foo;" \
>     | /opt/hypertable/current/bin/ht shell --batch
000	c1	Hypertable
000	c2	is
001	c1	very
001	c2	easy

如果你想清楚所有表运行下面命令

$ /opt/hypertable/current/bin/stop-servers.sh
$ /opt/hypertable/current/bin/clean-database.sh


38.2. Code examples

http://hypertable.com/documentation/code_examples/

38.2.1. PHP

设置环境变量

export PHPTHRIFT_ROOT=/opt/hypertable/current/lib/php

安装PHP环境

# yum install php-cli

建立测试文件

# vim lib/php/test.php
<?php
if (!isset($GLOBALS['THRIFT_ROOT']))
    $GLOBALS['THRIFT_ROOT'] = getenv('PHPTHRIFT_ROOT');

require_once dirname(__FILE__).'/ThriftClient.php';

$client = new Hypertable_ThriftClient("localhost", 38080);
$namespace = $client->namespace_open("");

echo "HQL examples\n";
print_r($client->hql_query($namespace, "show tables"));
print_r($client->hql_query($namespace, "select * from foo"));

运行测试程序

# php lib/php/test.php
HQL examples
Hypertable_ThriftGen_HqlResult Object
(
    [results] => Array
        (
            [0] => foo
        )

    [cells] =>
    [scanner] =>
    [mutator] =>
)
Hypertable_ThriftGen_HqlResult Object
(
    [results] =>
    [cells] => Array
        (
            [0] => Hypertable_ThriftGen_Cell Object
                (
                    [key] => Hypertable_ThriftGen_Key Object
                        (
                            [row] => 000
                            [column_family] => c1
                            [column_qualifier] =>
                            [timestamp] => 1361518099519878001
                            [revision] => 1361518099519878001
                            [flag] => 255
                        )

                    [value] => Hypertable
                )

            [1] => Hypertable_ThriftGen_Cell Object
                (
                    [key] => Hypertable_ThriftGen_Key Object
                        (
                            [row] => 000
                            [column_family] => c2
                            [column_qualifier] =>
                            [timestamp] => 1361518099519878002
                            [revision] => 1361518099519878002
                            [flag] => 255
                        )

                    [value] => is
                )

            [2] => Hypertable_ThriftGen_Cell Object
                (
                    [key] => Hypertable_ThriftGen_Key Object
                        (
                            [row] => 001
                            [column_family] => c1
                            [column_qualifier] =>
                            [timestamp] => 1361518099519878003
                            [revision] => 1361518099519878003
                            [flag] => 255
                        )

                    [value] => very
                )

            [3] => Hypertable_ThriftGen_Cell Object
                (
                    [key] => Hypertable_ThriftGen_Key Object
                        (
                            [row] => 001
                            [column_family] => c2
                            [column_qualifier] =>
                            [timestamp] => 1361518099519878004
                            [revision] => 1361518099519878004
                            [flag] => 255
                        )

                    [value] => easy
                )

        )

    [scanner] =>
    [mutator] =>
)


38.3. HQL

38.3.1. namespace 命名空间管理

hypertable> create namespace test;

  Elapsed time:  0.22 s

hypertable> use test;

  Elapsed time:  0.00 s

hypertable> drop namespace test;

  Elapsed time:  1.55 s

38.3.2. Table 表

创建表

CREATE TABLE logging (
    uuid,
    tag,
    asctime,
    facility,
    priority,
    message,
    operator
)

hypertable> CREATE TABLE logging (
         -> uuid,
         -> tag,
         -> asctime,
         -> facility,
         -> priority,
         -> message,
         -> operator
         -> );

  Elapsed time:  2.14 s

列出所有表

hypertable> show tables;
logging

  Elapsed time:  0.00 s
hypertable>

显示创建该表的HQL

hypertable> show create table logging;

CREATE TABLE logging (
  uuid,
  tag,
  asctime,
  facility,
  priority,
  message,
  operator,
  ACCESS GROUP default (uuid, tag, asctime, facility, priority, message, operator)
)

  Elapsed time:  0.00 s
hypertable>

删除表

hypertable> drop table logging;

  Elapsed time:  1.33 s


38.4. FAQ

38.4.1. 切换 DFS Broker

从local切换到ceph

/opt/hypertable/current/bin/stop-servers.sh ; /opt/hypertable/current/bin/start-dfsbroker.sh local ; /opt/hypertable/current/bin/clean-database.sh
rm -rf /opt/hypertable/current/hyperspace/* /opt/hypertable/current/fs/* /opt/hypertable/current/run/rsml_backup/* /opt/hypertable/current/run/last-dfs

启用ceph

# /opt/hypertable/current/bin/start-dfsbroker.sh ceph
DFS broker: available file descriptors: 1024
Waiting for DFS Broker (ceph) (localhost:38030) to come up...


第 39 章 Membase


第 40 章 TokyoCabinet/Tyrant

http://www.162cm.com/p/tokyotyrant.html

# yum install tokyocabinet tokyocabinet-devel -y


第 41 章 Flare


第 42 章 Voldemort


第 43 章 LevelDB

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

http://code.google.com/p/leveldb/


第 44 章 CouchDB


第 45 章 RethinkDB

http://www.rethinkdb.com/


第 46 章 HyperDex

http://hyperdex.org/


部分 VI. Search Engine

目录

47. Solr
47.1. Embedded Jetty
47.2. Jetty
47.3. Tomcat
47.4. solr-php-client
47.5. multicore
47.6. 中文分词
47.6.1. ChineseTokenizerFactory
47.6.2. CJK
47.6.3. mmseg4j
47.6.4. 中文分词“庖丁解牛” Paoding Analysis
48. Nutch
49. Sphinx
50. Lucene
51. MG4J
52. PhpDig
53. Mahout
54. elasticsearch


第 47 章 Solr

目录

47.1. Embedded Jetty
47.2. Jetty
47.3. Tomcat
47.4. solr-php-client
47.5. multicore
47.6. 中文分词
47.6.1. ChineseTokenizerFactory
47.6.2. CJK
47.6.3. mmseg4j
47.6.4. 中文分词“庖丁解牛” Paoding Analysis

http://lucene.apache.org/solr/

java 采用apt-get安装

例 47.1. /etc/profile.d/java.sh

################################################
### Java environment by neo
################################################
export JAVA_HOME=/usr
export JRE_HOME=/usr
export PATH=$PATH:/usr/local/apache-tomcat/bin/:/usr/local/jetty-6.1.18/bin
export CLASSPATH="./:/usr/share/java/:/usr/local/apache-solr/example/multicore/lib"
export JAVA_OPTS="-Xms128m -Xmx1024m"

47.1. Embedded Jetty

wget http://apache.freelamp.com/lucene/solr/1.3.0/apache-solr-1.3.0.tgz
tar zxvf apache-solr-1.3.0.tgz
ln -s apache-solr-1.3.0 ../apache-solr
cd ../apache-solr/example/
java -jar start.jar

multicore: java -Dsolr.solr.home=multicore -jar start.jar


47.2. Jetty

http://jetty.mortbay.org/jetty/

过程 47.1. apt-get install

  1. install

$ sudo apt-get install libxpp3-java
$ sudo apt-get install solr-jetty

  2. firewall

$ sudo ufw allow 8280

  3. Testing.

    http://172.16.0.1:8280/

    http://172.16.0.1:8280/admin/ (user:admin, passwd:admin)

过程 47.2. source codes install

  • download

wget http://dist.codehaus.org/jetty/jetty-6.1.18/jetty-6.1.18.zip


47.3. Tomcat

http://tomcat.apache.org/

  1. download

cd /usr/local/src

wget http://apache.etoak.com/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.tar.gz
wget http://apache.freelamp.com/lucene/solr/1.3.0/apache-solr-1.3.0.tgz

tar zxvf apache-tomcat-6.0.20.tar.gz
ln -s apache-tomcat-6.0.20 ../apache-tomcat

tar zxvf apache-solr-1.3.0.tgz
ln -s apache-solr-1.3.0 ../apache-solr

  2. solr.xml

vim /usr/local/apache-tomcat/conf/Catalina/localhost/solr.xml

<Context docBase="/usr/local/apache-solr/dist/apache-solr-1.3.0.war" debug="0" crossContext="true" >
   <Environment name="solr/home" type="java.lang.String" value="/usr/local/apache-solr/example/solr" override="true" />
</Context>


47.4. solr-php-client

http://code.google.com/p/solr-php-client/

wget http://solr-php-client.googlecode.com/files/SolrPhpClient.2009-03-11.tgz
tar zxvf SolrPhpClient.2009-03-11.tgz
sudo mv SolrPhpClient/Apache /usr/share/php/


47.5. multicore

solr.xml

vim /usr/local/apache-solr/example/multicore/solr.xml

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="false">
  <cores adminPath="/admin/cores">
    <core name="core0" instanceDir="core0" />
    <core name="core1" instanceDir="core1" />

    <core name="article" instanceDir="article" />

  </cores>
</solr>

core directory and config file

mkdir -p article/conf

vim article/conf/solrconfig.xml

<?xml version="1.0" encoding="UTF-8" ?>
<config>
  <updateHandler class="solr.DirectUpdateHandler2" />
  <requestDispatcher handleSelect="true" >
    <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
  </requestDispatcher>
  <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
  <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
  <requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
  <admin>
    <defaultQuery>solr</defaultQuery>
  </admin>
</config>

vim article/conf/schema.xml

<?xml version="1.0" ?>
<schema name="example core zero" version="1.1">
  <types>
   <fieldType name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/>
   <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
   <fieldType name="date" class="solr.DateField" sortMissingLast="true" omitNorms="true"/>
   <fieldType name="text" class="solr.TextField" positionIncrementGap="100" />
  </types>
 <fields>
  <!-- general -->
  <field name="id"      type="sint"   indexed="true"  stored="true"  multiValued="false" required="true"/>
  <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="title"   type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="content"   type="text"   indexed="true"  stored="true"  multiValued="false" />
  <field name="timestamp" type="date" indexed="true" stored="true" default="NOW"/>
 </fields>
 <!-- field to use to determine and enforce document uniqueness. -->
 <uniqueKey>id</uniqueKey>
 <!-- field for the QueryParser to use when an explicit fieldname is absent -->
 <defaultSearchField>content</defaultSearchField>
 <!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
 <solrQueryParser defaultOperator="OR"/>
    <copyField source="title" dest="content"/>
    <copyField source="name" dest="content"/>
</schema>

commit datas

vim test.xml

<add>
    <doc>
      <field name="id">1</field>
      <field name="name">Hello world</field>
    </doc>

    <doc>
      <field name="id">2</field>
      <field name="title">Title Hello world</field>
    </doc>

    <doc>
      <field name="id">3</field>
      <field name="name">Hello world 1</field>
      <field name="content">Content 1</field>
    </doc>

    <doc>
      <field name="id">4</field>
      <field name="name">Name Neo</field>
    </doc>

    <doc>
      <field name="id">5</field>
      <field name="name">Last Chan</field>
    </doc>
</add>

java -Durl=http://localhost:8983/solr/article/update -Dcommit=yes -jar ../exampledocs/post.jar test.xml


47.6. 中文分词

47.6.1. ChineseTokenizerFactory

<fieldType name="text" class="solr.TextField" >
    <analyzer>
        <tokenizer class="org.apache.solr.analysis.ChineseTokenizerFactory"/>
    </analyzer>
</fieldType>

47.6.2. CJK

<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
    <analyzer>
        <tokenizer class="solr.CJKTokenizerFactory"/>
    </analyzer>
</fieldType>

47.6.3. mmseg4j

http://code.google.com/p/mmseg4j/

install

$ cd /usr/local/src/
$ wget http://mmseg4j.googlecode.com/files/mmseg4j-1.7.2.zip
$ unzip mmseg4j-1.7.2.zip
$ mkdir /usr/local/apache-solr/example/multicore/lib
$ cp /usr/local/src/mmseg4j-1.7.2/mmseg4j-all-1.7.2.jar /usr/local/apache-solr/example/multicore/lib
$ cd mmseg4j-1.7.2/

test

$ java -Dmmseg.dic.path=/usr/local/apache-solr/example/solr -jar mmseg4j-all-1.7.2.jar 这里是字符串
$ java -Dmmseg.dic.path=/usr/local/apache-solr/example/solr -cp .:mmseg4j-all-1.7.2.jar com.chenlb.mmseg4j.example.Simple 这里是字符串
$ java -Dmmseg.dic.path=/usr/local/apache-solr/example/solr -cp .:mmseg4j-all-1.7.2.jar com.chenlb.mmseg4j.example.MaxWord 这里是字符串

mmseg4j 在 solr 中主要支持两个参数：mode、dicPath。mode 表示是什么模式分词（有效值：simplex、complex、max-word，如果输入了无效的默认用 max-word。）。dicPath 是词库目录可以是绝对目录，也可以是相对目录（是相对 solr.home 目录下的，dic 就会在 solr.home/dic 目录下找词库文件），如果不指定就是默认在 CWD/data 目录（程序运行当前目录的data子目录）下找。

分词例子

<fieldtype name="textComplex" class="solr.TextField">
    <analyzer>
        <tokenizer class="com.chenlb.mmseg4j.solr.MMSegTokenizerFactory"
            mode="complex" dicPath="dic">
        </tokenizer>
    </analyzer>
</fieldtype>

<fieldtype name="textMaxWord" class="solr.TextField">
    <analyzer>
        <tokenizer class="com.chenlb.mmseg4j.solr.MMSegTokenizerFactory"
            mode="max-word" dicPath="dic">
        </tokenizer>
    </analyzer>
</fieldtype>

<fieldtype name="textSimple" class="solr.TextField">
    <analyzer>
        <tokenizer class="com.chenlb.mmseg4j.solr.MMSegTokenizerFactory"
            mode="simple" dicPath="/usr/local/apache-solr/example/solr/my_dic">
        </tokenizer>
    </analyzer>
</fieldtype>

添加到schema.xml

<fieldType name="text" class="solr.TextField" positionIncrementGap="100" >
    <analyzer>
        <tokenizer class="com.chenlb.mmseg4j.solr.MMSegTokenizerFactory" mode="complex" dicPath="dic"/>
        <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
</fieldType>

http://localhost:8080/solr/admin/analysis.jsp 在 Field 的下拉菜单选择 name，然后在应用输入 complex。可以看 mmseg4j 的分词的结果.

47.6.4. 中文分词“庖丁解牛” Paoding Analysis

$ cd /usr/local/src/
$ mkdir paoding-analysis-2.0.4-beta
$ cd paoding-analysis-2.0.4-beta/
$ wget http://paoding.googlecode.com/files/paoding-analysis-2.0.4-beta.zip
$ unzip paoding-analysis-2.0.4-beta.zip
$ cp paoding-analysis.jar /usr/local/apache-solr/example/multicore/lib/

ChineseTokenizerFactory


第 48 章 Nutch

http://lucene.apache.org/nutch/

How to Setup Nutch and Hadoop

http://wiki.apache.org/nutch/NutchHadoopTutorial

  1. 下载

    $ cd /usr/local/src/
$ wget http://apache.etoak.com/lucene/nutch/nutch-1.0.tar.gz
$ tar zxvf nutch-1.0.tar.gz
$ sudo cp -r nutch-1.0 ..
$ cd ..
$ sudo ln -s nutch-1.0 apache-nutch

  2. 创建文件myurl

    $ cd apache-nutch
$ mkdir urls
$ vim urls/myurl
http://netkiller.8800.org/

  3. 配置文件 crawl-urlfilter.txt

    编辑conf/crawl-urlfilter.txt文件，修改MY.DOMAIN.NAME部分，把它替换为你想要抓取的域名

    $ cp conf/crawl-urlfilter.txt conf/crawl-urlfilter.txt.old
$ vim conf/crawl-urlfilter.txt

# accept hosts in MY.DOMAIN.NAME
+^http://([a-z0-9]*\.)*MY.DOMAIN.NAME/
修改为：
# accept hosts in MY.DOMAIN.NAME
+^http://([a-z0-9]*\.)*netkiller.8800.org/

  4. http.agent.name

$ vim conf/nutch-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>
  <name>http.agent.name</name>
  <value>Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5</value>
  <description>HTTP 'User-Agent' request header. MUST NOT be empty -
  please set this to a single word uniquely related to your organization.

  NOTE: You should also check other related properties:

    http.robots.agents
    http.agent.description
    http.agent.url
    http.agent.email
    http.agent.version

  and set their values appropriately.

  </description>
</property>

<property>
  <name>http.agent.description</name>
  <value></value>
  <description>Further description of our bot- this text is used in
  the User-Agent header.  It appears in parenthesis after the agent name.
  </description>
</property>

<property>
  <name>http.agent.url</name>
  <value>http://netkiller.8800.org/robot.html</value>
  <description>A URL to advertise in the User-Agent header.  This will
   appear in parenthesis after the agent name. Custom dictates that this
   should be a URL of a page explaining the purpose and behavior of this
   crawler.
  </description>
</property>

<property>
  <name>http.agent.email</name>
  <value>openunix@163.com</value>
  <description>An email address to advertise in the HTTP 'From' request
   header and User-Agent header. A good practice is to mangle this
   address (e.g. 'info at example dot com') to avoid spamming.
  </description>
</property>

</configuration>

  5. 运行以下命令行开始工作

    $ bin/nutch crawl urls -dir crawl -depth 3 -threads 5

bin/nutch crawl <your_url> -dir <your_dir> -depth 2 -threads 4 >&logs/logs1.log

urls 存放需要爬行的url文件的目录，即目录/nutch/urls。
-dir  dirnames    	设置保存所抓取网页的目录.
-depth  depth 		表明抓取网页的层次深度
-delay  delay		表明访问不同主机的延时，单位为“秒”
-threads  threads  	表明需要启动的线程数
-topN 50	topN	一个网站保存的最大页面数。

$ nohup bin/nutch crawl /usr/local/apache-nutch/urls -dir /usr/local/apache-nutch/crawl -depth 5 -threads 50 -topN 50 > /tmp/nutch.log &

  6. depoly

$ cd /usr/local/apache-tomcat/conf/Catalina/localhost
$ vim nutch.xml
<Context docBase="/usr/local/apache-nutch/nutch-1.0.war" debug="0" crossContext="true" >
</Context>

    searcher.dir

$ vim /usr/local/apache-tomcat/webapps/nutch/WEB-INF/classes/nutch-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>searcher.dir</name>
        <value>/usr/local/apache-nutch/crawl</value>
    </property>
</configuration>

    test

    http://172.16.0.1:8080/nutch/


第 49 章 Sphinx

http://sphinxsearch.com/

sudo apt-get install sphinxsearch

/etc/sphinxsearch/sphinx.conf

sudo cp /etc/sphinxsearch/sphinx-min.conf.dist /etc/sphinxsearch/sphinx.conf

创建测试数据库并导入测试数据

$ wget http://sphinxsearch.googlecode.com/svn/trunk/example.sql
$ mysql -h localhost -uroot -p < example.sql
$ mysql -h localhost -uroot -p
CREATE USER 'test'@'localhost' IDENTIFIED BY  '';
GRANT SELECT ON test.* TO 'test'@'localhost';
FLUSH PRIVILEGES;
mysql> quit

$ echo "select * from documents" | mysql -utest -p test
Enter password:
id      group_id        group_id2       date_added      title   content
1       1       5       2011-02-12 15:29:34     test one        this is my test document number one. also checking search within phrases.
2       1       6       2011-02-12 15:29:34     test two        this is my test document number two
3       2       7       2011-02-12 15:29:34     another doc     this is another group
4       2       8       2011-02-12 15:29:34     doc number four this is to test groups

创建索引

sudo indexer <index>

$ sudo indexer test1

Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file '/etc/sphinxsearch/sphinx.conf'...
indexing index 'test1'...
collected 4 docs, 0.0 MB
sorted 0.0 Mhits, 100.0% done
total 4 docs, 193 bytes
total 0.012 sec, 16531.05 bytes/sec, 342.61 docs/sec

$ sudo /etc/init.d/sphinxsearch start
Starting sphinx: Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file '/etc/sphinxsearch/sphinx.conf'...
creating server socket on 0.0.0.0:3312
sphinx.

测试

search "keyword"

$ search test
Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file '/etc/sphinxsearch/sphinx.conf'...
index 'test1': query 'test ': returned 3 matches of 3 total in 0.000 sec

displaying matches:
1. document=1, weight=2, group_id=1, date_added=Sat Feb 12 15:29:34 2011
        id=1
        group_id=1
        group_id2=5
        date_added=2011-02-12 15:29:34
        title=test one
        content=this is my test document number one. also checking search within phrases.
2. document=2, weight=2, group_id=1, date_added=Sat Feb 12 15:29:34 2011
        id=2
        group_id=1
        group_id2=6
        date_added=2011-02-12 15:29:34
        title=test two
        content=this is my test document number two
3. document=4, weight=1, group_id=2, date_added=Sat Feb 12 15:29:34 2011
        id=4
        group_id=2
        group_id2=8
        date_added=2011-02-12 15:29:34
        title=doc number four
        content=this is to test groups

words:
1. 'test': 3 documents, 5 hits

wget http://sphinxsearch.googlecode.com/svn/trunk/api/sphinxapi.php
wget http://sphinxsearch.googlecode.com/svn/trunk/api/test.php
php test.php test


第 50 章 Lucene

http://lucene.apache.org/


第 51 章 MG4J

http://mg4j.dsi.unimi.it/


第 52 章 PhpDig

http://www.phpdig.net/

PhpDig is a web spider and search engine written in PHP, using a MySQL database and flat file support. PhpDig builds a glossary with words found in indexed pages. On a search query, it displays a result page containing the search keys, ranked by occurrence.


第 53 章 Mahout

http://mahout.apache.org/


第 54 章 elasticsearch

http://www.elasticsearch.org/