部署k8s多节点
文章目錄
- k8s多節(jié)點(diǎn)
- 配置部署環(huán)境
- 負(fù)載均衡部署
- 部署keepalived服務(wù)
- k8s拓展知識(shí)
| Master01 | 192.168.136.88 | kube-apiserver kube-controller-manager kube-scheduler etcd |
| Master02 | 192.168.136.60 | kube-apiserver kube-controller-manager kube-scheduler |
| Ndoe01 | 192.168.136.40 | kubelet kube-proxy docker flannel etcd |
| Ndoe02 | 192.168.136.30 | kubelet kube-proxy docker flannel etcd |
| nginx01 | 192.168.136.10 | nginx、keepalived |
| nginx02 | 168.168.136.20 | nginx、keepalived |
k8s多節(jié)點(diǎn)
配置部署環(huán)境
優(yōu)先關(guān)閉防火墻和selinux服務(wù)
在master01上操作
復(fù)制kubernetes目錄到master02
[root@localhost ~]# scp -r /opt/kubernetes/ root@192.168.136.60:/opt復(fù)制master中的三個(gè)組件啟動(dòng)腳本到master02上
scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.136.60:/usr/lib/systemd/system/需要拷貝master01上已有的etcd證書(shū)給master02使用
特別注意：master02一定要有etcd證書(shū)
[root@localhost ~]# scp -r /opt/etcd/ root@192.168.136.60:/opt/master02上操作
修改配置文件
cd /opt/kubernetes/cfg/ [root@master2 cfg]# vim kube-apiserver5 --bind-address=192.168.136.60 \6 --secure-port=6443 \7 --advertise-address=192.168.136.60 \啟動(dòng)master02中的三個(gè)組件服務(wù)
[root@localhost cfg]# systemctl start kube-apiserver.service [root@localhost cfg]# systemctl start kube-controller-manager.service [root@localhost cfg]# systemctl start kube-scheduler.service增加環(huán)境變量
[root@localhost cfg]# vim /etc/profile #末尾添加 export PATH=$PATH:/opt/kubernetes/bin/ [root@matser02 cfg]# source /etc/profile檢查是否開(kāi)啟
[root@matser02 cfg]# kubectl get node NAME STATUS ROLES AGE VERSION 192.168.136.30 Ready <none> 24h v1.12.3 192.168.136.40 Ready <none> 25h v1.12.3負(fù)載均衡部署
在01和b02上操作
負(fù)載均衡
192.168.136.10",
192.168.136.20",
b01 和b02操作，建立nginx源
[root@localhost ~]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0安裝nginx
[root@localhost ~]# yum install nginx -y添加apr四層轉(zhuǎn)發(fā)
所謂四層就是基于IP+端口的負(fù)載均衡，通過(guò)虛擬IP+端口接收請(qǐng)求，然后再分配到真實(shí)的服務(wù)器
[root@localhost ~]# vim /etc/nginx/nginx.conf events {worker_connections 1024; } -----------------------------------添加 stream {log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';access_log /var/log/nginx/k8s-access.log main;upstream k8s-apiserver {server 192.168.136.80:6443;server 192.168.136.60:6443;}server {listen 6443;proxy_pass k8s-apiserver;}} ----------------------------------------- http {開(kāi)啟服務(wù)查看端口
[root@localhost ~]# systemctl start nginx [root@localhost ~]# netstat -ntap | grep nginx tcp 0 0 0.0.0.0:6443 0.0.0.0:* LISTEN 67128/nginx: master tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 67128/nginx: master部署keepalived服務(wù)
配置01和02節(jié)點(diǎn)
[root@localhost ~]# yum install keepalived -y修改配置文件nide1節(jié)點(diǎn)
刪除原有的配置文件，復(fù)制下面的代碼
[root@localhost ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { # 接收郵件地址 notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } # 郵件發(fā)送地址 notification_email_from Alexandre.Cassen@firewall.loc smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id NGINX_MASTER } vrrp_script check_nginx {script "/etc/nginx/check_nginx.sh" }vrrp_instance VI_1 { state MASTER interface ens33virtual_router_id 51 # VRRP 路由 ID實(shí)例，每個(gè)實(shí)例是唯一的 priority 100 # 優(yōu)先級(jí)，備服務(wù)器設(shè)置 90 advert_int 1 # 指定VRRP 心跳包通告間隔時(shí)間，默認(rèn)1秒 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.136.100/24 } track_script {check_nginx} }修改配置文件nide2節(jié)點(diǎn)
刪除原有的配置文件，復(fù)制下面的代碼
[root@localhost ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { # 接收郵件地址 notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } # 郵件發(fā)送地址 notification_email_from Alexandre.Cassen@firewall.loc smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id NGINX_MASTER } vrrp_script check_nginx {script "/etc/nginx/check_nginx.sh" }vrrp_instance VI_1 { state BACKUP interface ens33virtual_router_id 51 # VRRP 路由 ID實(shí)例，每個(gè)實(shí)例是唯一的 priority 90 # 優(yōu)先級(jí)，備服務(wù)器設(shè)置 90 advert_int 1 # 指定VRRP 心跳包通告間隔時(shí)間，默認(rèn)1秒 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.136.100/24 } track_script {check_nginx} }在fznode1和fznode2上建立nginx腳本（目的nginx關(guān)閉keepalived也隨之關(guān)閉）
[root@localhost ~]# vim /etc/nginx/check_nginx.sh count=$(ps -ef |grep nginx |egrep -cv "grep|$$")if [ "$count" -eq 0 ];thensystemctl stop keepalived fi加權(quán)限
[root@localhost ~]# chmod +x /etc/nginx/check_nginx.sh開(kāi)啟服務(wù)
[root@localhost ~]# systemctl start keepalived//查看lb01地址信息
驗(yàn)證地址漂移
（fz01中使用pkill nginx，再在fz02中使用ip a 查看）
//恢復(fù)操作（在lb01中先啟動(dòng)nginx服務(wù)，再啟動(dòng)keepalived服務(wù)）
開(kāi)始修改node節(jié)點(diǎn)配置文件統(tǒng)一VIP
注意不是負(fù)載均衡節(jié)點(diǎn)是下面👇，兩個(gè)都要改
node節(jié)點(diǎn)
192.168.136.40
192.168.136.30
開(kāi)啟服務(wù)
[root@localhost cfg]# systemctl restart kubelet.service [root@localhost cfg]# systemctl restart kube-proxy.service替換完成直接自檢
[root@localhost cfg]# grep 100 * bootstrap.kubeconfig: server: https://192.168.136.100:6443 kubelet.kubeconfig: server: https://192.168.136.100:6443 kube-proxy.kubeconfig: server: https://192.168.136.100:6443在lb01上查看nginx的k8s日志
[root@localhost ~]# tail /var/log/nginx/k8s-access.log 192.168.136.40 192.168.136.60:6443 - [05/Oct/2020:23:23:59 +0800] 200 1121 192.168.136.30 192.168.136.80:6443, 192.168.136.60:6443 - [05/Oct/2020:23:24:00 +0800] 200 0, 1119 192.168.136.40 192.168.136.80:6443, 192.168.136.60:6443 - [05/Oct/2020:23:24:00 +0800] 200 0, 1119 192.168.136.30 192.168.136.80:6443, 192.168.136.60:6443 - [05/Oct/2020:23:24:00 +0800] 200 0, 1121在master01上操作
測(cè)試創(chuàng)建pod
kubectl run nginx --image=nginx查看狀態(tài)
[root@localhost ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-nf9sk 0/1 ContainerCreating 0 33s //正在創(chuàng)建中 [root@localhost ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-nf9sk 1/1 Running 0 80s //創(chuàng)建完成，運(yùn)行中k8s拓展知識(shí)
k8s注意日志問(wèn)題
看不了日志因?yàn)槲覀兪悄涿?/p> [root@localhost ~]# kubectl logs nginx-dbddb74b8-c8gjh Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-c8gjh)
解決：
賦予管理員權(quán)限
kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous再次查看就可以看到
[root@localhost ~]# kubectl logs nginx-dbddb74b8-c8gjh /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf 10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Configuration complete; ready for start up查看日志
查看pod網(wǎng)絡(luò)（mster端口操作）
[root@localhost ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE nginx-dbddb74b8-c8gjh 1/1 Running 0 8m57s 172.17.54.2 192.168.136.40 <none>在對(duì)應(yīng)網(wǎng)段的node節(jié)點(diǎn)上操作可以直接訪問(wèn)
[root@localhost cfg]# curl 172.17.54.2 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style>body {width: 35em;margin: 0 auto;font-family: Tahoma, Verdana, Arial, sans-serif;} </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p><p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p> </body> </html>訪問(wèn)就會(huì)產(chǎn)生日志
[root@localhost ~]# kubectl get pods -o wide 172.17.54.1 - - [05/Oct/2020:15:39:24 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"總結(jié)
- 上一篇： 内存条频率：提速神器还是电脑加速利器？
- 下一篇： MHA+LAMP+Keepalived实