just do it

Redis集群方案redis-twemproxy-keepalived

redis现在的版本还不支持分布式,这会在3.0版本加上。Redis目前只支持“主从同步”和“从从同步”,所以需要根据需求应用自己来实现。

《Redis集群方案redis-twemproxy-keepalived》

 

tp —— 192.168.10.15

tp1—— 192.168.10.18

tp tp1 虚拟vip: 192.168.10.243

 

v1 —— 192.168.10.224

v2 —— 192.168.10.14

v1 v2 虚拟vip : 192.168.10.241

 

v3 —— 192.168.10.16

v4 —— 192.168.10.17

v3 v4 虚拟vip: 192.168.10.242

 

 

  1. 1.       先做redis 热备节点 2

 

R1 R2 为第一组节点

V1 (192.168.10.224)  V2 (192.168.10.14) 通过keepalived

 

(master)  V1

备份(backup)V2

虚拟ip  192.168.10.241

 

 

搭建步骤:主相同

      1. 安装redis

    

    wget http://redis.googlecode.com/files/redis-2.0.0-rc4.tar.gz

tar zxvf redis-2.0.0-rc4.tar.gz

cd redis-2.0.0-rc4

make

cp redis.conf /etc/ 这个文件是redis启动的配置文件

cp redis-benchmark redis-cli redis-server /usr/bin/

 

redis-server /etc/redis.conf

redis-cli

注意修改配置:daemonize yes 

 

      2. 安装高可用软件 keepalived

 

wget http://www.keepalived.org/software/keepalived-1.1.20.tar.gz

tar zxvf keepalived-1.1.20.tar.gz

cd keepalived-1.1.20

./configure –with-kernel-dir=/usr/src/kernels/2.6.18-164.el5-x86_64/

make && make install

 

cp /usr/local/etc/rc.d/init.d/keepalived  /etc/rc.d/init.d/

cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

mkdir /etc/keepalived

cp /usr/local/etc/keepalived/keepalived.conf  /etc/keepalived/

cp /usr/local/sbin/keepalived  /usr/sbin

/etc/init.d/keepalived start

 

 

报错:

!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!

解决

yum install openssl-devel

 

报错

configure: error: Popt libraries is required

解决

yum install popt-devel

 

 

 

配置文件修改注意所监控网卡硬件名称

 

V1配置文件

Keepalived:

 

state MASTER     

 

virtual_ipaddress {

             192.168.10.241  ###VIP

        }

 

 

redis_backup.sh :

 

SLAVEOF 192.168.10.14 6379

 

redis_master.sh:

 

SLAVEOF 192.168.10.14 6379

 

 

V2 配置文件

 

Keepalived:

 

 

state BACKUP     

 

virtual_ipaddress {

             192.168.10.241  ###VIP

        }

 

redis_backup.sh :

 

SLAVEOF 192.168.10.224  6379

 

redis_master.sh:

 

SLAVEOF 192.168.10.224 6379

 

 

测试此互备节点:

 

1.启动Master上的Redis

redis-server /etc/redis.conf

2.启动Slave上的Redis

redis-server /etc/redis.conf

3.启动Master上的Keepalived

/etc/init.d/keepalived start

4.启动Slave上的Keepalived

  /etc/init.d/keepalived start

5.尝试通过VIP连接Redis:

redis-cli -h 192.168.10.241 INFO

连接成功,Slave也连接上来了。

role:master

6.尝试插入一些数据:

redis-cli -h 192.168.10.241  SET Hellor1 Redis

OK

VIP读取数据

redis-cli -h 192.168.10.241  GET Hellor1

“Redis”

Master读取数据

redis-cli -h 192.168.10.224 GET Hellor1

“Redis”

Slave读取数据

  redis-cli -h 192.168.10.14 GET Hellor1

“Redis”

 

 

下面,模拟故障产生:

 

Master上的Redis进程杀死:

killall -9 redis-server

查看Master上的Keepalived日志

tailf /var/log/keepalived-redis-state.log

[fault]

Mon Dec 30 10:06:13 CST 2013

同时Slave上的日志显示:

tailf /var/log/keepalived-redis-state.log

[master]

Mon Dec 30 10:06:17 CST 2013

Being master….

Run SLAVEOF cmd …

OK

Run SLAVEOF NO ONE cmd …

OK

然后我们可以发现,Slave已经接管服务,并且担任Master的角色了。

redis-cli -h 192.168.10.241 INFO

redis-cli -h 192.168.10.14 INFO

role:master

然后我们恢复MasterRedis进程

redis-server /etc/redis.conf

查看Master上的Keepalived日志

tailf /var/log/keepalived-redis-state.log

[master]

Mon Dec 30 10:10:13 CST 2013

Being master….

Run SLAVEOF cmd …

OK

Run SLAVEOF NO ONE cmd …

OK

同时Slave上的日志显示:

tailf /var/log/keepalived-redis-state.log

[backup]

Mon Dec 30 10:10:12 CST 2013

Being slave….

Run SLAVEOF cmd …

OK

可以发现目前的Master已经再次恢复了Master的角色,故障切换以及自动恢复都成功了。

 

 

R3 R4 为第二组节点

参考第一组即可

 

V3(192.168.10.16)  V4 (192.168.10.17) 通过keepalived

 

(master)  V3

备份(backup)V4

虚拟ip  192.168.10.242

virtual_router_id 不同

 

 

 

 

  1. 2.       twemproxy 代理互备

 

 

Tp (192.168.10.15)

Tp1(192.168.10.18)

 

搭建步骤:

 

1.安装 twemproxy

 

 

yum install automake

yum install libtool

git clone git://github.com/twitter/twemproxy.git

cd twemproxy

 

CFLAGS=”-ggdb3 -O0″ autoreconf -fvi && ./configure –prefix=/usr/local/twemproxy  –enable-debug=log

 

编译的时候报错

 

autoreconf: Entering directory `.’ 

autoreconf: configure.ac: not using Gettext 

autoreconf: running: aclocal –force -I m4 

configure.ac:8: error: Autoconf version 2.64 or higher is required 

configure.ac:8: the top level 

autom4te: /usr/bin/m4 failed with exit status: 63 

aclocal: autom4te failed with exit status: 63 

autoreconf: aclocal failed with exit status: 63 

 

原因是autoconf版本过低

 

升级

查看当前版本

#rpm -qf /usr/bin/autoconf  

autoconf-2.63-5.1.el6.noarch

 

卸载当前版本

rpm -e –nodeps autoconf-2.63   

 

下载新版本

wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.64.tar.gz

 

 

解压安装

tar zxvf autoconf-2.64.tar.gz 

cd autoconf-2.64 

./configure –prefix=/usr 

make && make install 

 

查看是否安装成功

/usr/bin/autoconf -V

 

 

CFLAGS=”-ggdb3 -O0″ autoreconf -fvi && ./configure –prefix=/usr/local/twemproxy  –enable-debug=log

make && make install 

 

cp conf/nutcracker.yml /usr/local/twemproxy/conf/ 

 

测试

/usr/local/twemproxy/sbin/nutcracker -t

nutcracker: configuration file ‘conf/nutcracker.yml’ syntax is ok

 

————————————————————————–

启动命令

调试启动

 

/usr/local/twemproxy/sbin/nutcracker -c

 

以守护进程启动

/usr/local/twemproxy/sbin/nutcracker -d -c

 

 

2.安装 keepalived

参考上面安装步骤即可

 

 

修改配置文件

 

Tp 配置文件

 

Keepalived

 

vrrp_script chk_t {

                script “/etc/keepalived/scripts/t_check.sh”   ###监控脚本

                interval 2                                        ###监控时间

}

vrrp_instance VI_1 {

        state MASTER                            ###设置为MASTER

        interface eth1                          ###监控网卡

        virtual_router_id 53

        priority 101                            ###权重值

        authentication {

                     auth_type PASS             ###加密

                     auth_pass redis            ###密码

        }

        track_interface {

            eth1

        }

        virtual_ipaddress {

             192.168.10.243                         ###VIP

        }

        notify_master /etc/keepalived/scripts/t_master.sh

        notify_backup /etc/keepalived/scripts/t_backup.sh

        notify_fault  /etc/keepalived/scripts/t_fault.sh

        notify_stop   /etc/keepalived/scripts/t_stop.sh

}

 

/usr/local/twemproxy/conf/nutcracker.yml

redis:

  listen: 192.168.10.243:6379

  hash: fnv1a_64

  distribution: ketama

  auto_eject_hosts: true

  redis: true

  server_retry_timeout: 2000

  server_failure_limit: 1

  servers:

   – 192.168.10.241:6379:1 node1

   – 192.168.10.242:6379:1 node2

 

 

 

Tp1 配置文件

 

Keepalived

 

vrrp_script chk_t {

                script “/etc/keepalived/scripts/t_check.sh”   ###监控脚本

                interval 2                                        ###监控时间

}

vrrp_instance VI_1 {

        state BACKUP                            ###设置为BACKUP

        interface eth1                          ###监控网卡

        virtual_router_id 53

        priority 99                            ###权重值

        authentication {

                     auth_type PASS             ###加密

                     auth_pass redis            ###密码

        }

         track_interface {

            eth1

        }

        virtual_ipaddress {

             192.168.10.243                         ###VIP

        }

        notify_master /etc/keepalived/scripts/t_master.sh

        notify_backup /etc/keepalived/scripts/t_backup.sh

        notify_fault  /etc/keepalived/scripts/t_fault.sh

        notify_stop   /etc/keepalived/scripts/t_stop.sh

}

 

/usr/local/twemproxy/conf/nutcracker.yml

redis:

  listen: 192.168.10.243:6379

  hash: fnv1a_64

  distribution: ketama

  auto_eject_hosts: true

  redis: true

  server_retry_timeout: 2000

  server_failure_limit: 1

  servers:

   – 192.168.10.241:6379:1 node1

   – 192.168.10.242:6379:1 node2

 

测试

redis-cli -h 192.168.10.243

redis> set pp pp

OK

redis> get pp

“pp”

 

关掉tp 查看tp1 日志

tailf /var/log/keepalived-t-state.log

[backup]

Tue Dec 31 14:37:20 CST 2013

Being slave….

[master]

Tue Dec 31 14:47:28 CST 2013

Being master….

Run twemproxy …

 

Tp2 接管 vip 继续提供服务

redis-cli -h 192.168.10.243

redis> get pp

“pp”

 

重启tp 查看tp1 日志

tailf /var/log/keepalived-t-state.log

[backup]

Tue Dec 31 14:54:37 CST 2013

Being slave….

 

redis-cli -h 192.168.10.243

redis> get pp

“pp”

 Tp接管主服务 tp2切换备份服务

 

 
3.可能遇到相关问题和处理办法:
1. 如果两机器不能连通,请注意关掉防火墙: 
service iptables stop #临时关闭
chkconfig iptables off #永久关闭
2. windows上编写的shell脚本,要注意通过dos2unix 命令转换,否则shell脚本可能有不可见符号不能正常执行。

点赞