⼀、 Keeplive服务介绍
Keeplive期初是专门为LVS设计的，专门⽤来监控LVS集群系统中各个服务节点的状态，后来⼜加⼊VRRP的功能，因此除了配合LVS服务以外，也可以作为其他服务（nginx，haroxy）的⾼可⽤软件，VRRP是Virtual Router Redundancy Protocol（虚拟路由冗余协议）的缩写，VRRP出现的⽬的就是为了解决静态路由出现的单点故障问题，它能够保证⽹络的不间断，稳定的运⾏。所以keepalive⼀⽅⾯具有LVScluster nodes healthchecks功能，另⼀⽅⾯也具有LVS directors failover
1.1 Keepalived的⽤途
Keepalive服务的两⼤⽤途：healthcheck和failover
ha failover功能：实现LB Master主机和Backup主机之间故障转移和⾃动切换
这是针对有两个负载均衡器Direator同时⼯作⽽采取的故障转移措施，当主负载均衡器失效或者出现故障时，备份的负载均衡器
（BACKUP）将⾃动接管主负载均衡器的所有⼯作（vip资源以及相关服务）：⼀旦主负载均衡器故障恢复，MASTER⼜会接管回它原来的⼯作，⼆备份复杂均衡器（BACKUP）会释放master是⼩事它接管的⼯作，此时两者将恢复到最初各⾃的⾓⾊
1.2 LVS cluster nodes healthchecks功能
在keeplive.conf配置记忆可以实现LVS的功能keeplive可以对LVS下⾯的集群节点进⾏健康检查
rs healthcheck功能：负载均衡定期检查RS的可⽤性决定是否给其分发请求
当虚拟的服务器中的某⼀个甚⾄是⼏个真实的服务器同时出现故障⽆法提供服务时，负载均衡器会⾃动将失效的RS服务器从转发队列中清除出去，从⽽保证⽤户的访问不收影响；当故障的RS服务器被修复后，系统⼜⾃动的将他们加⼊转发队列，分发请求提供正常服务。⼯作原理
1.3 keepalive故障切换转换原理
Keepalived⾼可⽤对之间是通过 VRRP进⾏通信的， VRRP是遑过竞选机制来确定主备的，主的优先级⾼于备，因此，⼯作时主会优先获得所有的资源，备节点处于等待状态，当主挂了的时候，备节点就会接管主节点的资源，然后顶替主节点对外提供服务。
在 Keepalived服务对之间，只有作为主的服务器会⼀直发送 VRRP⼴播包,告诉备它还活着，此时备不会占主，当主不可⽤时，即备监听不到主发送的⼴播包时，就会启动相关服务接管资源，保证业务的连续性.接管速度最快可以⼩于1秒。
1.4 VRRP协议的简单介绍
1) VRRP,全称 Virtual Router Redundancy Protocol,中⽂名为虚拟路由冗余协议，VRRP的出现是为了解决静态路由的单点故障。2) VRRP是通过⼀种竟选协议机制来将路由任务交给某台 VRRP路由器的。3) VRRP⽤ IP多播的⽅式（默认多播地址（224.0_0.18))实现⾼可⽤对之间通信。
4) ⼯作时主节点发包，备节点接包，当备节点接收不到主节点发的数据包的时候，就启动接管程序接管主节点的开源。备节点可以有多个，通过优先级竞选，但⼀般 Keepalived系统运维⼯作中都是⼀对。
5) VRRP使⽤了加密协议加密数据，但Keepalived官⽅⽬前还是推荐⽤明⽂的⽅式配置认证类型和密码
⼆、配置Keepalived实现⾼可⽤
2.1 安装keepalive
[root@lb01 ~]# cd /usr/local/src/
[root@lb01 src]# tar -xf keepalived-2.0.15.tar.gz[root@lb01 src]# cd keepalived-2.0.15[root@lb01 keepalived-2.0.15]# ./configure
Linker flags : -pie
Extra Lib : -lm -lcrypto -lssl Use IPVS Framework : YesIPVS use libnl : No
IPVS syncd attributes : NoIPVS bit stats : No
HTTP_GET regex support : Nofwmark socket support : YesUse VRRP Framework : YesUse VRRP VMAC : YesUse VRRP authentication : YesWith ip rules/routes : YesWith track_process : YesWith linkbeat : YesUse BFD Framework : NoSNMP vrrp support : NoSNMP checker support : NoSNMP RFCv2 support : NoSNMP RFCv3 support : NoDBUS support : NoSHA1 support : NoUse JSON output : Nolibnl version : NoneUse IPv4 devconf : NoUse iptables : YesUse libiptc : NoUse libipset : NoUse nftables : Noinit type : systemdStrict config checks : NoBuild genhash : YesBuild documentation : No
[root@lb01 keepalived-2.0.15]# make[root@lb01 keepalived-2.0.15]# make installlb02相同操作
[root@lb02 ~]# cd /usr/local/src/
[root@lb02 src]# wget https://www.keepalived.org/software/keepalived-2.0.15.tar.gz[root@lb02 src]# tar -xf keepalived-2.0.15.tar.gz[root@lb02 src]# cd keepalived-2.0.15
[root@lb02 keepalived-2.0.15]# ./configure [root@lb02 keepalived-2.0.15]# make
[root@lb02 keepalived-2.0.15]# make install
[root@lb01 keepalived-2.0.15]# vim /usr/lib/systemd/system/keepalived.service [Unit]
Description=LVS and VRRP High Availability MonitorAfter=syslog.target network-online.target
[Service]Type=forking
PIDFile=/var/run/keepalived.pidKillMode=process
EnvironmentFile=-/etc/sysconfig/keepalived
ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONSExecReload=/bin/kill -HUP $MAINPID[Install]
WantedBy=multi-user.target
2.2 配置⽂件
[root@lb01 keepalived-2.0.15]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived #！注释
global_defs { #全局变量 notification_email {
283365585@qq.com #收件⼈ } #邮件发件⼈
notification_email_from Alexandre.Cassen@firewall.loc smtp_server 192.168.200.1 #邮件服务器地址 smtp_connect_timeout 30 #超时时间 router_id LVS_01
vrrp_skip_check_adv_addr vrrp_strict
vrrp_garp_interval 0 vrrp_gna_interval 0}
vrrp_instance VI_1 { #keepalive或者vrrp的⼀个实例 state MASTER #状态 interface ens33 #通信端⼝ virtual_router_id 51 #实例ID priority 150 #优先级
advert_int 1 #⼼跳的间隔
authentication { #服务器之间通过密码验证 auth_type PASS auth_pass 1111 }
virtual_ipaddress {
10.0.0.131 #VIP } }
2.3 启动看效果
[root@lb01 keepalived]# systemctl start keepalived[root@lb01 keepalived]# ps -ef|grep keep[root@lb01 keepalived]# ip addr
1: lo: valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: inet 172.25.2.131/24 brd 172.25.2.255 scope global dynamic ens33 valid_lft 1085sec preferred_lft 1085sec inet 10.0.0.131/24 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::8068:96e2:b57b:be1d/ scope link valid_lft forever preferred_lft forever 3: ens34: 2.4 keepalive的VIP实现形式 [root@lb01 keepalived]# ifconfig ens33:0 10.0.0.18 up[root@lb01 keepalived]# ip addr add 10.0.0.19 dev ens33 [root@lb01 keepalived]# ip addr inet 172.25.2.131/24 brd 172.25.2.255 scope global dynamic ens33 valid_lft 1583sec preferred_lft 1583sec inet 10.0.0.131/24 scope global ens33 valid_lft forever preferred_lft forever inet 10.0.0.18/8 brd 10.255.255.255 scope global ens33:0 valid_lft forever preferred_lft forever inet 10.0.0.19/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::8068:96e2:b57b:be1d/ scope link valid_lft forever preferred_lft forever [root@lb01 keepalived]# ip addr del 10.0.0.19 dev ens33[root@lb01 keepalived]# ifconfig ens33:0 10.0.0.18 down [root@lb01 keepalived]# scp /etc/keepalived/keepalived.conf 172.25.2.133:/etc/keepalived/[root@lb02 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 283365585@qq.com } notification_email_from Alexandre.Cassen@firewall.loc smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id LVS_02 vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0} vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.131/24 }} [root@lb02 ~]# systemctl start keepalived[root@lb02 ~]# ps -ef |grep keep配置成功 2.5 检测keepalibve效果 关闭MASTER的keepalive服务 [root@lb01 keepalived]# systemctl stop keepalived[root@lb01 keepalived]# ip addr|grep 10.0.0.131查看BACKUP端，是否有10.0.0.131[root@lb02 ~]# ip addr|grep 10.0.0.131 [root@lb01 keepalived]# systemctl start keepalived成功 三、 结合nginx实现⾼可⽤ 3.1 配置 [root@lb01 keepalived]# cd /usr/local/nginx/conf/[root@lb01 conf]# vim nginx.conf worker_processes 1;events { worker_connections 1024;} http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65;upstream web_pools { server 172.25.2.134:80 weight=5; server 172.25.2.135:80 weight=5; # server 172.25.2.158:80 weight=5 backup;} server { listen 80; server_name www.lbtest.com; location / { # root html; # index index.html index.htm; proxy_set_header Host $host; proxy_pass http://web_pools; } }} [root@lb01 conf]# scp nginx.conf 172.25.2.133:/usr/local/nginx/conf/ [root@lb01 conf]# nginx -s reload[root@lb01 conf]# curl 172.25.2.134172.25.2.134 [root@lb01 conf]# curl 172.25.2.135172.25.2.135 [root@lb01 conf]# nginx -s reload[root@lb02 ~]# curl 172.25.2.134172.25.2.134 [root@lb02 ~]# curl 172.25.2.135172.25.2.135 在获取到VIP后，不在同⼀⽹段，为了⽅便测试，把VIP设置为172.25.2.2[root@lb01 ~]# ip addr|grep 172.25.2.2 3.2 发现不能访问的问题 访问测试，发现不能访问，也不能ping通[root@lb01 conf]# curl 172.25.2.2 curl: (7) Failed connect to 172.25.2.2:80; Connection timed out [root@lb01 conf]# ping 172.25.2.2 2 packets transmitted, 0 received, 100% packet loss, time 999ms 解决 [root@lb01 conf]# vim /etc/keepalived/keepalived.conf # vrrp_strict # 注释掉vrrp_strict [root@lb01 conf]# systemctl restart keepalived 3.3 测试 [root@lb01 conf]# curl 172.25.2.2172.25.2.135 [root@lb01 conf]# curl 172.25.2.2172.25.2.134 [root@lb01 conf]# curl 172.25.2.2172.25.2.135 [root@lb01 conf]# curl 172.25.2.2172.25.2.134 关闭MASTER测试 [root@lb01 conf]# systemctl stop keepalived #这时VIP已经在BACKUP上，但是可以正常访问[root@lb01 conf]# curl 172.25.2.2172.25.2.135 [root@lb01 conf]# curl 172.25.2.2172.25.2.134 [root@lb01 conf]# curl 172.25.2.2172.25.2.135 [root@lb01 conf]# curl 172.25.2.2172.25.2.134 [root@lb01 conf]# curl 172.25.2.2172.25.2.135 使⽤主机，域名访问 172.25.2.2 www.lbtest.com #写hosts 正常访问 四、keepalived的其他特性 4.1 监控⾃动迁移脚本 keepalived解决的是主机级别的冗余，当nginx宕掉的时候，keepalive并不会迁移，这时VIP依然在该主机上，客户就不能访问到⽹站使⽤脚本监控，当nginx挂掉，⾃动停掉keepalive，是VIP漂移，是业务不受影响 [root@lb01 conf]# mkdir /script[root@lb01 conf]# vim /script/monitor.sh #!/bin/bashwhile truedo if [ `ps -ef |grep nginx|grep -v grep |wc -l` -lt 2 ] then systemctl stop keepalivedfi sleep 5done [root@lb01 conf]# cd /script/ [root@lb01 script]# chmod +x monitor.sh [root@lb01 script]# /script/monitor.sh &关掉nginx [root@lb01 script]# nginx -s stopVIP漂移到BACKUP上 [root@lb02 ~]# ip addr|grep 2.2 inet 172.25.2.2/24 scope global secondary ens33 访问： [root@lb01 script]# curl 172.25.2.2172.25.2.134 [root@lb01 script]# curl 172.25.2.2172.25.2.135 [root@lb01 script]# curl 172.25.2.2172.25.2.134 [root@lb01 script]# curl 172.25.2.2172.25.2.135 4.2 keepalive⾼可⽤脑裂脚本 [root@lb01 script]# ps -ef |grep monitor root 80993 68563 0 07:04 pts/0 00:00:00 /bin/bash /script/monitor.sh[root@lb01 script]# kill -9 80993 [1]+ Killed /script/monitor.sh[root@lb01 script]# ps -ef |grep monitor root 82773 68563 0 07:13 pts/0 00:00:00 grep --color=auto monitor[root@lb01 script]# systemctl restart keepalived [root@lb02 ~]# mkdir /script 检测脑裂脚本，实现形式，当BACKUP可以ping通主，但是VIP依然在BACKUP，则认定为脑裂[root@lb02 script]# vim check_split_brain.sh #!/bin/bashwhile truedo ping -c 2 -W 3 172.25.2.131 &>/dev/null if [ $? -eq 0 -a `ip addr|grep 172.25.2.2|wc -l` -eq 1 ] then echo \"ha is split brain warning\"else echo \"ha is OK\"fi sleep 3done [root@lb02 ~]# sh /script/check_split_brain.sh ha is OKha is OK [root@lb02 ~]# systemctl start firewalldha is split brain warning ha is split brain warning [root@lb02 ~]# ip addr |grep 172.25.2.2 inet 172.25.2.2/24 scope global secondary ens33[root@lb01 ~]# ip addr |grep 172.25.2.2 inet 172.25.2.2/24 scope global secondary ens33 [root@node4 ~]# systemctl stop firewalldha is OKha is OK 4.3 修改⽇志⽂件路径 配置⽂件默认在/var/log/messages [root@lb01 ~]# tail -f /var/log/messages Apr 13 07:41:26 node2 Keepalived_vrrp[82796]: Sending gratuitous ARP on ens33 for 172.25.2.2Apr 13 07:41:26 node2 Keepalived_vrrp[82796]: Sending gratuitous ARP on ens33 for 172.25.2.2Apr 13 07:41:26 node2 Keepalived_vrrp[82796]: Sending gratuitous ARP on ens33 for 172.25.2.2 Apr 13 07:41:26 node2 Keepalived_vrrp[82796]: VRRP_Instance(VI_1) Received advert with lower priority 100, ours 150, forcing new electionApr 13 07:41:26 node2 Keepalived_vrrp[82796]: Sending gratuitous ARP on ens33 for 172.25.2.2 Apr 13 07:41:26 node2 Keepalived_vrrp[82796]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.25.2.2 修改⽇志⽂件位置 [root@lb01 ~]# vi /etc/sysconfig/keepalived # Options for keepalived. See `keepalived --help' output and keepalived(8) and# keepalived.conf(5) man pages for a list of all options. Here are the most# common ones :# # --vrrp -P Only run with VRRP subsystem. # --check -C Only run with Health-checker subsystem. # --dont-release-vrrp -V Dont remove VRRP VIPs & VROUTEs on daemon stop.# --dont-release-ipvs -I Dont remove IPVS topology on daemon stop.# --dump-conf -d Dump the configuration data.# --log-detail -D Detailed log messages. # --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON)# KEEPALIVED_OPTIONS=\"-D -S 0 -d\" [root@lb01 ~]# vim /etc/rsyslog.conf local0.* /var/log/keepalived.log [root@lb01 ~]# systemctl restart rsyslog[root@lb01 ~]# systemctl restart keepalived[root@lb01 ~]# tail -f /var/log/keepalived.log Apr 13 07:46:23 node2 Keepalived_vrrp[84692]: Sending gratuitous ARP on ens33 for 172.25.2.2Apr 13 07:46:23 node2 Keepalived_vrrp[84692]: Sending gratuitous ARP on ens33 for 172.25.2.2Apr 13 07:46:23 node2 Keepalived_vrrp[84692]: Sending gratuitous ARP on ens33 for 172.25.2.2 Apr 13 07:46:23 node2 Keepalived_vrrp[84692]: VRRP_Instance(VI_1) Received advert with lower priority 100, ours 150, forcing new electionApr 13 07:46:23 node2 Keepalived_vrrp[84692]: Sending gratuitous ARP on ens33 for 172.25.2.2 Apr 13 07:46:23 node2 Keepalived_vrrp[84692]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.25.2.2 4.4 keepalived多实例 [root@lb01 ~]# vim /etc/keepalived/keepalived.conf 1 ! Configuration File for keepalived 2 3 global_defs { 4 notification_email { 5 283365585@qq.com 6 } 7 notification_email_from Alexandre.Cassen@firewall.loc 8 smtp_server 192.168.200.1 9 smtp_connect_timeout 30 10 router_id LVS_01 11 vrrp_skip_check_adv_addr 12 # vrrp_strict 13 vrrp_garp_interval 0 14 vrrp_gna_interval 0 15 } 16 17 vrrp_instance VI_1 { 18 state MASTER 19 interface ens33 20 virtual_router_id 51 21 priority 150 22 advert_int 1 23 authentication { 24 auth_type PASS 25 auth_pass 1111 26 } 27 virtual_ipaddress { 28 172.25.2.2/24 29 } 30 } 31 vrrp_instance VI_2 { 32 state BACKUP 33 interface ens33 34 virtual_router_id 52 35 priority 100 36 advert_int 1 37 authentication { 38 auth_type PASS 39 auth_pass 1111 40 } 41 virtual_ipaddress { 42 172.25.2.253/24 43 } 44 } [root@lb02 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 283365585@qq.com } notification_email_from Alexandre.Cassen@firewall.loc smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id LVS_02 vrrp_skip_check_adv_addr # vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0} vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.2.2/24 }} vrrp_instance VI_2 { state MASTER interface ens33 virtual_router_id 52 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.2.253/24 }} 检测 [root@lb01 ~]# systemctl restart keepalived[root@lb02 ~]# systemctl restart keepalived[root@lb01 ~]# ip addr inet 172.25.2.131/24 brd 172.25.2.255 scope global dynamic ens33 valid_lft 499sec preferred_lft 499sec inet 172.25.2.2/24 scope global secondary ens33 valid_lft forever preferred_lft forever[root@lb02 ~]# ip addr inet 172.25.2.133/24 brd 172.25.2.255 scope global dynamic ens33 valid_lft 422sec preferred_lft 422sec inet 172.25.2.253/24 scope global secondary ens33 valid_lft forever preferred_lft forever[root@lb02 ~]# systemctl stop keepalived[root@lb02 ~]# ip addr|grep 172.25.2 inet 172.25.2.133/24 brd 172.25.2.255 scope global dynamic ens33[root@lb01 ~]# ip addr |grep 172.25.2 inet 172.25.2.131/24 brd 172.25.2.255 scope global dynamic ens33 inet 172.25.2.2/24 scope global secondary ens33 inet 172.25.2.253/24 scope global secondary ens33[root@lb02 ~]# systemctl start keepalived[root@lb02 ~]# ip addr|grep 172.25.2 inet 172.25.2.133/24 brd 172.25.2.255 scope global dynamic ens33 inet 172.25.2.253/24 scope global secondary ens33[root@lb01 ~]# ip addr |grep 172.25.2 inet 172.25.2.131/24 brd 172.25.2.255 scope global dynamic ens33 inet 172.25.2.2/24 scope global secondary ens33 因篇幅问题不能全部显示，请点此查看更多更全内容
Copyright © 2019- dfix.cn 版权所有 湘ICP备2024080961号-1
违法及侵权请联系：TEL:199 1889 7713 E-MAIL:2724546146@qq.com
本站由北京市万商天勤律师事务所王兴未律师提供法律服务