Redis集群方案redis-twemproxy-keepalived

linux / 2013年12月31日 15时14分 / 15732人浏览
redis现在的版本还不支持分布式,这会在3.0版本加上。Redis目前只支持“主从同步”和“从从同步”,所以需要根据需求应用自己来实现。 20131231150905   tp ------ 192.168.10.15 tp1------ 192.168.10.18 tp tp1 虚拟vip: 192.168.10.243   v1 ------ 192.168.10.224 v2 ------ 192.168.10.14 v1 v2 虚拟vip : 192.168.10.241   v3 ------ 192.168.10.16 v4 ------ 192.168.10.17 v3 v4 虚拟vip: 192.168.10.242    
  1. 1.       先做redis 热备节点 2
  R1 R2 为第一组节点 V1 (192.168.10.224)  V2 (192.168.10.14) 通过keepalived   (master)  V1 备份(backup)V2 虚拟ip  192.168.10.241     搭建步骤:主相同       1. 安装redis          wget http://redis.googlecode.com/files/redis-2.0.0-rc4.tar.gz tar zxvf redis-2.0.0-rc4.tar.gz cd redis-2.0.0-rc4 make cp redis.conf /etc/ 这个文件是redis启动的配置文件 cp redis-benchmark redis-cli redis-server /usr/bin/   redis-server /etc/redis.conf redis-cli 注意修改配置:daemonize yes          2. 安装高可用软件 keepalived   wget http://www.keepalived.org/software/keepalived-1.1.20.tar.gz tar zxvf keepalived-1.1.20.tar.gz cd keepalived-1.1.20 ./configure --with-kernel-dir=/usr/src/kernels/2.6.18-164.el5-x86_64/ make && make install   cp /usr/local/etc/rc.d/init.d/keepalived  /etc/rc.d/init.d/ cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ mkdir /etc/keepalived cp /usr/local/etc/keepalived/keepalived.conf  /etc/keepalived/ cp /usr/local/sbin/keepalived  /usr/sbin /etc/init.d/keepalived start     报错: !!! OpenSSL is not properly installed on your system. !!! !!! Can not include OpenSSL headers files. !!! 解决 yum install openssl-devel   报错 configure: error: Popt libraries is required 解决 yum install popt-devel       配置文件修改注意所监控网卡硬件名称   V1配置文件 Keepalived:   state MASTER        virtual_ipaddress {              192.168.10.241  ###VIP         }     redis_backup.sh :   SLAVEOF 192.168.10.14 6379   redis_master.sh:   SLAVEOF 192.168.10.14 6379     V2 配置文件   Keepalived:     state BACKUP        virtual_ipaddress {              192.168.10.241  ###VIP         }   redis_backup.sh :   SLAVEOF 192.168.10.224  6379   redis_master.sh:   SLAVEOF 192.168.10.224 6379     测试此互备节点:   1.启动Master上的Redis redis-server /etc/redis.conf 2.启动Slave上的Redis redis-server /etc/redis.conf 3.启动Master上的Keepalived /etc/init.d/keepalived start 4.启动Slave上的Keepalived   /etc/init.d/keepalived start 5.尝试通过VIP连接Redis: redis-cli -h 192.168.10.241 INFO 连接成功,Slave也连接上来了。 role:master 6.尝试插入一些数据: redis-cli -h 192.168.10.241  SET Hellor1 Redis OK VIP读取数据 redis-cli -h 192.168.10.241  GET Hellor1 "Redis" Master读取数据 redis-cli -h 192.168.10.224 GET Hellor1 "Redis" Slave读取数据   redis-cli -h 192.168.10.14 GET Hellor1 "Redis"     下面,模拟故障产生:   Master上的Redis进程杀死: killall -9 redis-server 查看Master上的Keepalived日志 tailf /var/log/keepalived-redis-state.log [fault] Mon Dec 30 10:06:13 CST 2013 同时Slave上的日志显示: tailf /var/log/keepalived-redis-state.log [master] Mon Dec 30 10:06:17 CST 2013 Being master.... Run SLAVEOF cmd ... OK Run SLAVEOF NO ONE cmd ... OK 然后我们可以发现,Slave已经接管服务,并且担任Master的角色了。 redis-cli -h 192.168.10.241 INFO redis-cli -h 192.168.10.14 INFO role:master 然后我们恢复MasterRedis进程 redis-server /etc/redis.conf 查看Master上的Keepalived日志 tailf /var/log/keepalived-redis-state.log [master] Mon Dec 30 10:10:13 CST 2013 Being master.... Run SLAVEOF cmd ... OK Run SLAVEOF NO ONE cmd ... OK 同时Slave上的日志显示: tailf /var/log/keepalived-redis-state.log [backup] Mon Dec 30 10:10:12 CST 2013 Being slave.... Run SLAVEOF cmd ... OK 可以发现目前的Master已经再次恢复了Master的角色,故障切换以及自动恢复都成功了。     R3 R4 为第二组节点 参考第一组即可   V3(192.168.10.16)  V4 (192.168.10.17) 通过keepalived   (master)  V3 备份(backup)V4 虚拟ip  192.168.10.242 virtual_router_id 不同        
  1. 2.       twemproxy 代理互备
    Tp (192.168.10.15) Tp1(192.168.10.18)   搭建步骤:   1.安装 twemproxy     yum install automake yum install libtool git clone git://github.com/twitter/twemproxy.git cd twemproxy   CFLAGS="-ggdb3 -O0" autoreconf -fvi && ./configure --prefix=/usr/local/twemproxy  --enable-debug=log   编译的时候报错   autoreconf: Entering directory `.'  autoreconf: configure.ac: not using Gettext  autoreconf: running: aclocal --force -I m4  configure.ac:8: error: Autoconf version 2.64 or higher is required  configure.ac:8: the top level  autom4te: /usr/bin/m4 failed with exit status: 63  aclocal: autom4te failed with exit status: 63  autoreconf: aclocal failed with exit status: 63    原因是autoconf版本过低   升级 查看当前版本 #rpm -qf /usr/bin/autoconf   autoconf-2.63-5.1.el6.noarch   卸载当前版本 rpm -e --nodeps autoconf-2.63      下载新版本 wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.64.tar.gz     解压安装 tar zxvf autoconf-2.64.tar.gz  cd autoconf-2.64  ./configure --prefix=/usr  make && make install    查看是否安装成功 /usr/bin/autoconf -V     CFLAGS="-ggdb3 -O0" autoreconf -fvi && ./configure --prefix=/usr/local/twemproxy  --enable-debug=log make && make install    cp conf/nutcracker.yml /usr/local/twemproxy/conf/    测试 /usr/local/twemproxy/sbin/nutcracker -t nutcracker: configuration file 'conf/nutcracker.yml' syntax is ok   -------------------------------------------------------------------------- 启动命令 调试启动   /usr/local/twemproxy/sbin/nutcracker -c   以守护进程启动 /usr/local/twemproxy/sbin/nutcracker -d -c     2.安装 keepalived 参考上面安装步骤即可     修改配置文件   Tp 配置文件   Keepalived   vrrp_script chk_t {                 script "/etc/keepalived/scripts/t_check.sh"   ###监控脚本                 interval 2                                        ###监控时间 } vrrp_instance VI_1 {         state MASTER                            ###设置为MASTER         interface eth1                          ###监控网卡         virtual_router_id 53         priority 101                            ###权重值         authentication {                      auth_type PASS             ###加密                      auth_pass redis            ###密码         }         track_interface {             eth1         }         virtual_ipaddress {              192.168.10.243                         ###VIP         }         notify_master /etc/keepalived/scripts/t_master.sh         notify_backup /etc/keepalived/scripts/t_backup.sh         notify_fault  /etc/keepalived/scripts/t_fault.sh         notify_stop   /etc/keepalived/scripts/t_stop.sh }   /usr/local/twemproxy/conf/nutcracker.yml redis:   listen: 192.168.10.243:6379   hash: fnv1a_64   distribution: ketama   auto_eject_hosts: true   redis: true   server_retry_timeout: 2000   server_failure_limit: 1   servers:    - 192.168.10.241:6379:1 node1    - 192.168.10.242:6379:1 node2       Tp1 配置文件   Keepalived   vrrp_script chk_t {                 script "/etc/keepalived/scripts/t_check.sh"   ###监控脚本                 interval 2                                        ###监控时间 } vrrp_instance VI_1 {         state BACKUP                            ###设置为BACKUP         interface eth1                          ###监控网卡         virtual_router_id 53         priority 99                            ###权重值         authentication {                      auth_type PASS             ###加密                      auth_pass redis            ###密码         }          track_interface {             eth1         }         virtual_ipaddress {              192.168.10.243                         ###VIP         }         notify_master /etc/keepalived/scripts/t_master.sh         notify_backup /etc/keepalived/scripts/t_backup.sh         notify_fault  /etc/keepalived/scripts/t_fault.sh         notify_stop   /etc/keepalived/scripts/t_stop.sh }   /usr/local/twemproxy/conf/nutcracker.yml redis:   listen: 192.168.10.243:6379   hash: fnv1a_64   distribution: ketama   auto_eject_hosts: true   redis: true   server_retry_timeout: 2000   server_failure_limit: 1   servers:    - 192.168.10.241:6379:1 node1    - 192.168.10.242:6379:1 node2   测试 redis-cli -h 192.168.10.243 redis> set pp pp OK redis> get pp "pp"   关掉tp 查看tp1 日志 tailf /var/log/keepalived-t-state.log [backup] Tue Dec 31 14:37:20 CST 2013 Being slave.... [master] Tue Dec 31 14:47:28 CST 2013 Being master.... Run twemproxy ...   Tp2 接管 vip 继续提供服务 redis-cli -h 192.168.10.243 redis> get pp "pp"   重启tp 查看tp1 日志 tailf /var/log/keepalived-t-state.log [backup] Tue Dec 31 14:54:37 CST 2013 Being slave....   redis-cli -h 192.168.10.243 redis> get pp "pp"  Tp接管主服务 tp2切换备份服务     3.可能遇到相关问题和处理办法: 1. 如果两机器不能连通,请注意关掉防火墙:  service iptables stop #临时关闭 chkconfig iptables off #永久关闭 2. windows上编写的shell脚本,要注意通过dos2unix 命令转换,否则shell脚本可能有不可见符号不能正常执行。