Keepalived+Nginx实现高可用负载均衡集群
一 環(huán)境介紹
1.操作系統(tǒng)
CentOS Linux release 7.2.1511 (Core)
2.服務(wù)
keepalived+nginx雙主高可用負(fù)載均衡集群及LAMP應(yīng)用
keepalived-1.2.13-7.el7.x86_64
nginx-1.10.2-1.el7.x86_64
httpd-2.4.6-45.el7.centos.x86_64
二 原理及拓?fù)鋱D
1.vrrp協(xié)議
在現(xiàn)實(shí)的網(wǎng)絡(luò)環(huán)境中,兩臺(tái)需要通信的主機(jī)大多數(shù)情況下并沒有直接的物理連接。對于這樣的情況,它們之間路由怎樣選擇?主機(jī)如何選定到達(dá)目的主機(jī)的下一跳路由,這個(gè)問題通常的解決方法有二種:
?在主機(jī)上使用動(dòng)態(tài)路由協(xié)議(RIP、OSPF等)?
?在主機(jī)上配置靜態(tài)路由?
很明顯,在主機(jī)上配置動(dòng)態(tài)路由是非常不切實(shí)際的,因?yàn)楣芾怼⒕S護(hù)成本以及是否支持等諸多問題。配置靜態(tài)路由就變得十分流行,但路由器(或者說默認(rèn)網(wǎng)關(guān)default gateway)卻經(jīng)常成為單點(diǎn)故障。VRRP的目的就是為了解決靜態(tài)路由單點(diǎn)故障問題,VRRP通過一競選(election)協(xié)議來動(dòng)態(tài)的將路由任務(wù)交給LAN中虛擬路由器中的某臺(tái)VRRP路由器。
2.nginx反代
nginx是以反向代理的方式進(jìn)行負(fù)載均衡的。反向代理(Reverse Proxy)方式是指以代理服務(wù)器來接受Internet上的連接請求,然后將請求轉(zhuǎn)發(fā)給內(nèi)部網(wǎng)絡(luò)上的服務(wù)器,并將從服務(wù)器上得到的結(jié)果返回給Internet上請求連接的客戶端,此時(shí)代理服務(wù)器對外就表現(xiàn)為一個(gè)服務(wù)器。(為了理解反向代理,這里插播一條什么是正向代理:正向代理指的是,一個(gè)位于客戶端和原始服務(wù)器之間的服務(wù)器,為了從原始服務(wù)器取得內(nèi)容,客戶端向代理發(fā)送一個(gè)請求并指定目標(biāo)(原始服務(wù)器),然后代理向原始服務(wù)器轉(zhuǎn)交請求并將獲得的內(nèi)容返回給客戶端。)
3.拓?fù)鋱D
三 配置
1.后端RS配置
| 12 | [root@inode4 ~]# yum install httpd -y[root@inode5 ~]# yum install httpd -y |
?
?
2.Nginx反代配置
MASTER:
| 12345678910 | upstream websrvs {?server 172.18.67.11:80;?server 172.18.67.12:80;?server 127.0.0.1:80 backup;}server {????listen?????? 80 ;????location / {????proxy_pass http://websrvs;????} |
?
BACKUP:
| 12345678910 | upstream websrvs {?server 172.18.67.11:80;?server 172.18.67.12:80;?server 127.0.0.1:80 backup;}server {????listen?????? 80 ;????location / {????proxy_pass http://websrvs;????} |
?
3.keepalived高可用配置
MASTER:
| 1234567891011121314151617181920212223242526272829303132333435363738394041424344 | ! Configuration File?for?keepalivedglobal_defs {????notification_email {?root@localhost????}????notification_email_from keepalived@localhost????smtp_server 127.0.0.1????smtp_connect_timeout 30????router_id node1????vrrp_mcast_group4 224.0.67.67}vrrp_script chk_down {????script?"[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"????interval 1????weight -5}vrrp_script chk_nginx {????script?"killall -0 nginx && exit 0 || exit 1"????interval 1????weight -5????fall 2????rise 1}vrrp_instance myr {????state MASTER????interface eno16777736????virtual_router_id 167????priority 100????advert_int 1????authentication {?auth_type PASS?auth_pass 571f97b2????}????virtual_ipaddress {?172.18.67.33/16?dev eno16777736????}????track_script {?chk_down?chk_nginx????}????notify_master?"/etc/keepalived/notify.sh master"????notify_backup?"/etc/keepalived/notify.sh backup"????notify_fault?"/etc/keepalived/notify.sh fault"} |
?
BACKUP:
| 1234567891011121314151617181920212223242526272829303132333435363738394041424344 | ! Configuration File?for?keepalivedglobal_defs {????notification_email {?root@localhost????}????notification_email_from keepalived@localhost????smtp_server 127.0.0.1????smtp_connect_timeout 30????router_id node1????vrrp_mcast_group4 224.0.67.67}vrrp_script chk_down {????script?"[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"????interval 1????weight -5}vrrp_script chk_nginx {????script?"killall -0 nginx && exit 0 || exit 1"????interval 1????weight -5????fall 2????rise 1}vrrp_instance myr {????state BACKUP????interface eno16777736????virtual_router_id 167????priority 95????advert_int 1????authentication {?auth_type PASS?auth_pass 571f97b2????}????virtual_ipaddress {?172.18.67.33/16?dev eno16777736????}????track_script {?chk_down?chk_nginx????}????notify_master?"/etc/keepalived/notify.sh master"????notify_backup?"/etc/keepalived/notify.sh backup"????notify_fault?"/etc/keepalived/notify.sh fault"} |
?
4.通知腳本示例
| 123456789101112131415161718192021222324 | [root@inode2 nginx]# vim notify.sh#!/bin/bash#contact='root@localhost'notify() {?mailsubject="$(hostname) to be $1, vip floating"?mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"?echo?"$mailbody"?| mail -s?"$mailsubject"?$contact}case?$1?inmaster)?notify master?;;backup)?notify backup?;;fault)?notify fault?;;*)?echo?"Usage: $(basename $0) {master|backup|fault}"?exit?1?;;esac |
節(jié)點(diǎn)二同樣配置
四 啟動(dòng)服務(wù)并測試
1.啟動(dòng)后端web服務(wù)器
| 12 | [root@inode4 ~]# systemctl start httpd[root@inode5 ~]# systemctl start httpd |
為了測試顯示效果明顯一點(diǎn),自定義一個(gè)訪問頁面
| 12 | [root@inode4 ~]# echo "RS1:172.18.67.11" > /var/www/html/index.html[root@inode5 ~]# echo "RS2:172.18.67.12" > /var/www/html/index.html |
?
2.測試
MASTER:
| 123456789101112131415161718192021222324252627282930313233343536 | [root@inode2 ~]# systemctl start? keepalived[root@inode2 ~]# systemctl status -l? keepalived● keepalived.service - LVS and VRRP High Availability Monitor???Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)???Active: active (running) since Mon 2017-05-15 15:45:20 CST; 3s ago??Process: 20971 ExecStart=/usr/sbin/keepalived?$KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)?Main PID: 20972 (keepalived)???CGroup:?/system.slice/keepalived.service???????????├─20972?/usr/sbin/keepalived?-D???????????├─20973?/usr/sbin/keepalived?-D???????????└─20974?/usr/sbin/keepalived?-DMay 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Opening?file?'/etc/keepalived/keepalived.conf'.May 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Configuration is using : 7521 BytesMay 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Using LinkWatch kernel netlink reflector...May 15 15:45:20 inode2 Keepalived_vrrp[20974]: VRRP_Script(chk_nginx) succeededMay 15 15:45:21 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Transition to MASTER STATEMay 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Entering MASTER STATEMay 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) setting protocol VIPs.May 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736?for?172.18.67.33May 15 15:45:22 inode2 Keepalived_vrrp[20974]: Opening script?file?/etc/keepalived/notify.shMay 15 15:45:22 inode2 Keepalived_healthcheckers[20973]: Netlink reflector reports IP 172.18.67.33 added[root@inode2 ~]# ip a l1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN????link/loopback?00:00:00:00:00:00 brd 00:00:00:00:00:00????inet 127.0.0.1/8?scope host lo???????valid_lft forever preferred_lft forever????inet6 ::1/128?scope host???????valid_lft forever preferred_lft forever2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000????link/ether?00:0c:29:8b:08:6f brd ff:ff:ff:ff:ff:ff????inet 172.18.67.13/16?brd 172.18.255.255 scope global eno16777736???????valid_lft forever preferred_lft forever????inet 172.18.67.33/16?scope global secondary eno16777736???????valid_lft forever preferred_lft forever????inet6 fe80::20c:29ff:fe8b:86f/64?scope link tentative dadfailed???????valid_lft forever preferred_lft forever |
主節(jié)點(diǎn)啟動(dòng),Entering MASTER STATE,此時(shí)我們在客戶端進(jìn)行測試訪問
| 12345 | [root@inode1 ~]# for i in {1..4};do curl http://172.18.67.33;doneRS1:172.18.67.11RS2:172.18.67.12RS1:172.18.67.11RS2:172.18.67.12 |
?
訪問正常,接下來我們啟動(dòng)備用節(jié)點(diǎn)的服務(wù)器
BACKUP:
| 12345678910111213141516171819202122232425262728293031323334 | [root@inode3 keepalived]# systemctl start keepalived[root@inode3 keepalived]# systemctl status -l keepalived● keepalived.service - LVS and VRRP High Availability Monitor???Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)???Active: active (running) since Mon 2017-05-15 15:46:51 CST; 3s ago??Process: 24329 ExecStart=/usr/sbin/keepalived?$KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)?Main PID: 24330 (keepalived)???CGroup:?/system.slice/keepalived.service???????????├─24330?/usr/sbin/keepalived?-D???????????├─24331?/usr/sbin/keepalived?-D???????????└─24332?/usr/sbin/keepalived?-DMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: Registering Kernel netlink?command?channelMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: Registering gratuitous ARP shared channelMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: Opening?file?'/etc/keepalived/keepalived.conf'.May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Configuration is using : 66427 BytesMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: Using LinkWatch kernel netlink reflector...May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Entering BACKUP STATEMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: Opening script?file?/etc/keepalived/notify.shMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_down) succeededMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_nginx) succeeded[root@inode3 keepalived]# ip a l1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN????link/loopback?00:00:00:00:00:00 brd 00:00:00:00:00:00????inet 127.0.0.1/8?scope host lo???????valid_lft forever preferred_lft forever????inet6 ::1/128?scope host???????valid_lft forever preferred_lft forever2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000????link/ether?00:0c:29:78:24:c3 brd ff:ff:ff:ff:ff:ff????inet 172.18.67.14/16?brd 172.18.255.255 scope global eno16777736???????valid_lft forever preferred_lft forever????inet6 fe80::20c:29ff:fe78:24c3/64?scope link tentative dadfailed???????valid_lft forever preferred_lft forever |
?
此時(shí),我們可以看到備用節(jié)點(diǎn)服務(wù)器啟動(dòng)后進(jìn)入了BACKUP狀態(tài),Entering BACKUP STATE。接下來我們測試主節(jié)點(diǎn)宕機(jī)的情形下,我們的服務(wù)是否還可用
| 1 | [root@inode2 ~]# systemctl stop keepalived |
主節(jié)點(diǎn)宕機(jī)后我們查看備用節(jié)點(diǎn)的狀態(tài)
| 1234567891011121314151617181920212223242526272829303132333435 | [root@inode3 keepalived]# systemctl status -l keepalived● keepalived.service - LVS and VRRP High Availability Monitor???Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)???Active: active (running) since Mon 2017-05-15 15:46:51 CST; 2min 19s ago??Process: 24329 ExecStart=/usr/sbin/keepalived?$KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)?Main PID: 24330 (keepalived)???CGroup:?/system.slice/keepalived.service???????????├─24330?/usr/sbin/keepalived?-D???????????├─24331?/usr/sbin/keepalived?-D???????????└─24332?/usr/sbin/keepalived?-DMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_down) succeededMay 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_nginx) succeededMay 15 15:48:35 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Transition to MASTER STATEMay 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Entering MASTER STATEMay 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) setting protocol VIPs.May 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736?for?172.18.67.33May 15 15:48:36 inode3 Keepalived_vrrp[24332]: Opening script?file?/etc/keepalived/notify.shMay 15 15:48:36 inode3 Keepalived_healthcheckers[24331]: Netlink reflector reports IP 172.18.67.33 addedMay 15 15:48:41 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736?for?172.18.67.33[root@inode3 keepalived]# ip a l1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN????link/loopback?00:00:00:00:00:00 brd 00:00:00:00:00:00????inet 127.0.0.1/8?scope host lo???????valid_lft forever preferred_lft forever????inet6 ::1/128?scope host???????valid_lft forever preferred_lft forever2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000????link/ether?00:0c:29:78:24:c3 brd ff:ff:ff:ff:ff:ff????inet 172.18.67.14/16?brd 172.18.255.255 scope global eno16777736???????valid_lft forever preferred_lft forever????inet 172.18.67.33/16?scope global secondary eno16777736???????valid_lft forever preferred_lft forever????inet6 fe80::20c:29ff:fe78:24c3/64?scope link tentative dadfailed???????valid_lft forever preferred_lft forever |
?
我們發(fā)現(xiàn)備用節(jié)點(diǎn)由備用狀態(tài)進(jìn)入了主狀態(tài),并且IP地址也成功綁定至備用節(jié)點(diǎn)下。再次進(jìn)行測試訪問
| 12345 | [root@inode1 ~]# for i in {1..4};do curl http://172.18.67.33;doneRS1:172.18.67.11RS2:172.18.67.12RS1:172.18.67.11RS2:172.18.67.12 |
測試一臺(tái)web服務(wù)器宕機(jī)
| 123456 | [root@inode4 ~]# systemctl stop httpd[root@inode1 ~]# for i in {1..4};do curl http://172.18.67.33;doneRS2:172.18.67.12RS2:172.18.67.12RS2:172.18.67.12RS2:172.18.67.12 |
在實(shí)際生產(chǎn)環(huán)境中后端兩臺(tái)web服務(wù)器的內(nèi)容應(yīng)該一樣的,在這里我們可認(rèn)為客戶端已成功訪問到服務(wù)器,因此我們可認(rèn)為這樣的架構(gòu)體現(xiàn)了高可用負(fù)載均衡。
?
總結(jié)
以上是生活随笔為你收集整理的Keepalived+Nginx实现高可用负载均衡集群的全部內(nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 深入理解 Java 垃圾回收机制
- 下一篇: Nginx安装环境配置