雖然網(wǎng)上有大量從零搭建?K8S?的文章，但大都針對(duì)老版本，若直接照搬去安裝最新的?1.20?版本會(huì)遇到一堆問(wèn)題。故此將我的安裝步驟記錄下來(lái)，希望能為讀者提供?copy and paste?式的集群搭建幫助。

歡迎關(guān)注微信公眾號(hào)【廈門(mén)微思網(wǎng)絡(luò)】。www.xmws.cn專(zhuān)業(yè)IT認(rèn)證培訓(xùn)19周年
主要課程：思科、華為、紅帽、ORACLE、VMware、CISP、PMP等認(rèn)證培訓(xùn)及考證

?

?

1. 部署準(zhǔn)備工作

部署最小化 K8S 集群：master + node1 + node2

Ubuntu?是一款基于?Debian Linux?的以桌面應(yīng)用為主的操作系統(tǒng)，內(nèi)容涵蓋文字處理、電子郵件、軟件開(kāi)發(fā)工具和?Web?服務(wù)等，可供用戶免費(fèi)下載、使用和分享。

??vgs Current?machine?states: master????????????????????running?(virtualbox) node1?????????????????????running?(virtualbox) node2?????????????????????running?(virtualbox)

1.1 基礎(chǔ)環(huán)境信息

• 設(shè)置系統(tǒng)主機(jī)名以及 Host 文件各節(jié)點(diǎn)之間的相互解析

  • 使用這個(gè)的?Vagrantfile?啟動(dòng)的三節(jié)點(diǎn)服務(wù)已經(jīng)配置好了

  • 以下使用?master?節(jié)點(diǎn)進(jìn)行演示查看，其他節(jié)點(diǎn)操作均一致

#?hostnamectl vagrant@k8s-master:~$?hostnamectlStatic?hostname:?k8s-master#?hosts vagrant@k8s-master:~$?cat?/etc/hosts 127.0.0.1????????localhost 127.0.1.1????????vagrant.vm????vagrant 192.168.30.30????k8s-master 192.168.30.31????k8s-node1 192.168.30.32????k8s-node2#?ping vagrant@k8s-master:~$?ping?k8s-node1 PING?k8s-node1?(192.168.30.31)?56(84)?bytes?of?data. 64?bytes?from?k8s-node1?(192.168.30.31):?icmp_seq=1?ttl=64?time=0.689?ms

1.2 阿里源配置

• 配置 Ubuntu 的阿里源來(lái)加速安裝速度

  • 阿里源鏡像地址

#?登錄服務(wù)器 ??vgssh?master/node1/nod2 Welcome?to?Ubuntu?18.04.2?LTS?(GNU/Linux?4.15.0-50-generic?x86_64)#?設(shè)置阿里云Ubuntu鏡像 $?sudo?cp?/etc/apt/sources.list{,.bak} $?sudo?vim?/etc/apt/sources.list#?配置kubeadm的阿里云鏡像源 $?sudo?vim?/etc/apt/sources.list deb?https://mirrors.aliyun.com/kubernetes/apt?kubernetes-xenial?main $?sudo?gpg?--keyserver?keyserver.ubuntu.com?--recv-keys?BA07F4FB $?sudo?gpg?--export?--armor?BA07F4FB?|?sudo?apt-key?add?-#?配置docker安裝 $?curl?-fsSL?https://download.docker.com/linux/ubuntu/gpg?|?sudo?apt-key?add?- $?sudo?apt-key?fingerprint?0EBFCD88 $?sudo?vim?/etc/apt/sources.list deb?[arch=amd64]?https://download.docker.com/linux/ubuntu?bionic?stable#?更新倉(cāng)庫(kù) $?sudo?apt?update $?sudo?apt?dist-upgrade

1.3 基礎(chǔ)工具安裝

• 部署階段的基礎(chǔ)工具安裝

  • 基礎(chǔ)組件?docker

  • 部署工具?kubeadm

  • 路由規(guī)則?ipvsadm

  • 時(shí)間同步?ntp

#?基礎(chǔ)工具安裝 $?sudo?apt?install?-y?\docker-ce?docker-ce-cli?containerd.io?\kubeadm?ipvsadm?\ntp?ntpdate?\nginx?supervisor#?將當(dāng)前普通用戶加入docker組(需重新登錄) $?sudo?usermod?-a?-G?docker?$USER#?服務(wù)啟用 $?sudo?systemctl?enable?docker.service $?sudo?systemctl?start?docker.service $?sudo?systemctl?enable?kubelet.service $?sudo?systemctl?start?kubelet.service

1.4 操作系統(tǒng)配置

• 操作系統(tǒng)相關(guān)配置

  • 關(guān)閉緩存

  • 配置內(nèi)核參數(shù)

  • 調(diào)整系統(tǒng)時(shí)區(qū)

  • 升級(jí)內(nèi)核版本(默認(rèn)為4.15.0的版本)

#?關(guān)閉緩存 $?sudo?swapoff?-a#?為K8S來(lái)調(diào)整內(nèi)核參數(shù) $?sudo?touch?/etc/sysctl.d/kubernetes.conf $?sudo?cat?>?/etc/sysctl.d/kubernetes.conf?<<EOF net.bridge.bridge-nf-call-iptables?=?1?#?開(kāi)啟網(wǎng)橋模式(必須) net.bridge.bridge-nf-call-ip6tables?=?1?#?開(kāi)啟網(wǎng)橋模式(必須) net.ipv6.conf.all.disable_ipv6?=?1?#?關(guān)閉IPv6協(xié)議(必須) net.ipv4.ip_forward?=?1?#?轉(zhuǎn)發(fā)模式(默認(rèn)開(kāi)啟) vm.panic_on_oom=0?#?開(kāi)啟OOM(默認(rèn)開(kāi)啟) vm.swappiness?=?0?#?禁止使用swap空間 vm.overcommit_memory=1?#?不檢查物理內(nèi)存是否夠用 fs.inotify.max_user_instances=8192 fs.inotify.max_user_watches=1048576 fs.file-max?=?52706963?#?設(shè)置文件句柄數(shù)量 fs.nr_open?=?52706963?#?設(shè)置文件的最大打開(kāi)數(shù)量 net.netfilter.nf_conntrack_max?=?2310720 EOF#?查看系統(tǒng)內(nèi)核參數(shù)的方式 $?sudo?sysctl?-a?|?grep?xxx #?使內(nèi)核參數(shù)配置文件生效 $?sudo?sysctl?-p?/etc/sysctl.d/kubernetes.conf #?設(shè)置系統(tǒng)時(shí)區(qū)為中國(guó)/上海 $?sudo?timedatectl?set-timezone?Asia/Shanghai#?將當(dāng)前的UTC時(shí)間寫(xiě)入硬件時(shí)鐘 $?sudo?timedatectl?set-local-rtc?0

1.5 開(kāi)啟 ipvs 服務(wù)

• 開(kāi)啟 ipvs 服務(wù)

  • kube-proxy?開(kāi)啟?ipvs?的前置條件

#?載入指定的個(gè)別模塊 $?modprobe?br_netfilter#?修改配置 $?cat?>?/etc/sysconfig/modules/ipvs.modules?<<EOF #!/bin/bash modprobe?--?ip_vs modprobe?--?ip_vs_rr modprobe?--?ip_vs_wrr modprobe?--?ip_vs_sh modprobe?--?nf_conntrack_ipv EOF#?加載配置 $?chmod?755?/etc/sysconfig/modules/ipvs.modules?\&&?bash?/etc/sysconfig/modules/ipvs.modules?\&&?lsmod?|?grep?-e?ip_vs?-e?nf_conntrack_ipv

2. 部署 Master 節(jié)點(diǎn)

節(jié)點(diǎn)最低配置:?2C+2G?內(nèi)存；從節(jié)點(diǎn)資源盡量充足

kubeadm?工具的?init?命令，即可初始化以單節(jié)點(diǎn)部署的?master。為了避免翻墻，這里可以使用阿里云的谷歌源來(lái)代替。在執(zhí)行?kubeadm?部署命令的時(shí)候，指定對(duì)應(yīng)地址即可。當(dāng)然，可以將其加入本地的鏡像庫(kù)之中，更易維護(hù)。

• 注意事項(xiàng)

  • 阿里云谷歌源地址

  • 使用 kubeadm 定制控制平面配置

#?登錄服務(wù)器 ??vgssh?master Welcome?to?Ubuntu?18.04.2?LTS?(GNU/Linux?4.15.0-50-generic?x86_64)#?部署節(jié)點(diǎn)(命令行) #?注意pod和service的地址需要不同(否則會(huì)報(bào)錯(cuò)) $?sudo?kubeadm?init?\--kubernetes-version=1.20.2?\--image-repository?registry.aliyuncs.com/google_containers?\--apiserver-advertise-address=192.168.30.30?\--pod-network-cidr=10.244.0.0/16?\--service-cidr=10.245.0.0/16#?部署鏡像配置(配置文件) $?sudo?kubeadm?init?--config?./kubeadm-config.yaml Your?Kubernetes?control-plane?has?initialized?successfully! #?查看IP段是否生效(iptable) $?ip?route?show 10.244.0.0/24?dev?cni0?proto?kernel?scope?link?src?10.244.0.1 10.244.1.0/24?via?10.244.1.0?dev?flannel.1?onlink 10.244.2.0/24?via?10.244.2.0?dev?flannel.1?onlink#?#?查看IP段是否生效(ipvs) $?ipvsadm?-L?-n IP?Virtual?Server?version?1.2.1?(size=4096) Prot?LocalAddress:Port?Scheduler?Flags->?RemoteAddress:Port???????????Forward?Weight?ActiveConn?InActConn

• 配置文件定義

  • 接口使用了?v1beta2?版本

  • 配置主節(jié)點(diǎn)?IP?地址為?192.168.30.30

  • 為?flannel?分配的是?10.244.0.0/16?網(wǎng)段

  • 選擇的?kubernetes?是當(dāng)前最新的?1.20.2?版本

  • 加入了?controllerManager?的水平擴(kuò)容功能

#?kubeadm-config.yaml #?sudo?kubeadm?config?print?init-defaults?>?kubeadm-config.yaml apiVersion:?kubeadm.k8s.io/v1beta2 imageRepository:?registry.aliyuncs.com/google_containers kind:?ClusterConfiguration kubernetesVersion:?v1.20.2 apiServer:extraArgs:advertise-address:?192.168.30.30 networking:podSubnet:?10.244.0.0/16 controllerManager:ExtraArgs:horizontal-pod-autoscaler-use-rest-clients:?"true"horizontal-pod-autoscaler-sync-period:?"10s"node-monitor-grace-period:?"10s"

• 執(zhí)行成功之后會(huì)輸出如下信息，需要安裝如下步驟操作下

  • 第一步?在?kubectl?默認(rèn)控制和操作集群節(jié)點(diǎn)的時(shí)候，需要使用到?CA?的密鑰，傳輸過(guò)程是通過(guò)?TLS?協(xié)議保障通訊的安全性。通過(guò)下面?3?行命令拷貝密鑰信息到當(dāng)前用戶家目錄下，這樣?kubectl?執(zhí)行時(shí)會(huì)首先訪問(wèn)?.kube?目錄，使用這些授權(quán)信息訪問(wèn)集群。

  • 第二步?之后添加?worker?節(jié)點(diǎn)時(shí)，要通過(guò)?token?才能保障安全性。因此，先把顯示的這行命令保存下來(lái)，以備后續(xù)使用會(huì)用到。

#?master?setting?step?one To?start?cluster,?you?need?to?run?the?following?as?a?regular?user:mkdir?-p?$HOME/.kubesudo?cp?-i?/etc/kubernetes/admin.conf?$HOME/.kube/configsudo?chown?$(id?-u):$(id?-g)?$HOME/.kube/configAlternatively,?if?you?are?the?root?user,?you?can?run:export?KUBECONFIG=/etc/kubernetes/admin.conf#?master?setting?step?two You?should?now?deploy?a?pod?network?to?the?cluster. Run?"kubectl?apply?-f?[podnetwork].yaml"?with?one?of?the?options?listed:https://kubernetes.io/docs/concepts/cluster-administration/addons/Join?any?number?of?worker?nodes?by?running?the?following?on?each?as?root: kubeadm?join?192.168.30.30:6443?\--token?lebbdi.p9lzoy2a16tmr6hq?\--discovery-token-ca-cert-hash?\sha256:6c79fd83825d7b2b0c3bed9e10c428acf8ffcd615a1d7b258e9b500848c20cae

• 將子節(jié)點(diǎn)加入主節(jié)點(diǎn)中

$?kubectl?get?nodes NAME?????????STATUS?????ROLES??????????????????AGE???VERSION k8s-master???NotReady???control-plane,master???62m???v1.20.2 k8s-node1????NotReady???<none>?????????????????82m???v1.20.2 k8s-node2????NotReady???<none>?????????????????82m???v1.20.2 #?查看token令牌 $?sudo?kubeadm?token?list#?生成token令牌 $?sudo?kubeadm?token?create#?忘記sha編碼 $?openssl?x509?-pubkey?-in?/etc/kubernetes/pki/ca.crt?\|?openssl?rsa?-pubin?-outform?der?2>/dev/null?\|?openssl?dgst?-sha256?-hex?|?sed?'s/^.*?//' #?生成一個(gè)新的?token?令牌(比上面的方便) $?kubeadm?token?generate#?直接生成?join?命令(比上面的方便) $?kubeadm?token?create?<token_generate>?--print-join-command?--ttl=0

• 執(zhí)行完成之后可以通過(guò)如下命令，查看主節(jié)點(diǎn)信息

  • 默認(rèn)生成四個(gè)命名空間

• default、kube-system、kube-public、kube-node-lease

• 部署的核心服務(wù)有以下幾個(gè) (kube-system)

• coredns、etcd

• kube-apiserver、kube-scheduler

• kube-controller-manager、kube-controller-manager

• 此時(shí)?master?并沒(méi)有?ready?狀態(tài)(需要安裝網(wǎng)絡(luò)插件)，下一章節(jié)中，我們將安裝?flannel?這個(gè)網(wǎng)絡(luò)插件

#?命名空間 $?kubectl?get?namespace NAME??????????????STATUS???AGE default???????????Active???19m kube-node-lease???Active???19m kube-public???????Active???19m kube-system???????Active???19m#?核心服務(wù) $?kubectl?get?pod?-n?kube-system NAME?????????????????????????????????READY???STATUS????RESTARTS???AGE coredns-7f89b7bc75-bh42f?????????????1/1?????Running???0??????????19m coredns-7f89b7bc75-dvzpl?????????????1/1?????Running???0??????????19m etcd-k8s-master??????????????????????1/1?????Running???0??????????19m kube-apiserver-k8s-master????????????1/1?????Running???0??????????19m kube-controller-manager-k8s-master???1/1?????Running???0??????????19m kube-proxy-5rlpv?????????????????????1/1?????Running???0??????????19m kube-scheduler-k8s-master????????????1/1?????Running???0??????????19m

3. 部署 flannel 網(wǎng)絡(luò)

網(wǎng)絡(luò)服務(wù)用于管理 K8S 集群中的服務(wù)網(wǎng)絡(luò)

flannel?網(wǎng)絡(luò)需要指定?IP?地址段，即上一步中通過(guò)編排文件設(shè)置的?10.244.0.0/16。其實(shí)可以通過(guò)?flannel?官方和?HELM?工具直接部署服務(wù)，但是原地址是需要搭梯子的。所以，可以將其內(nèi)容保存在如下配置文件中，修改對(duì)應(yīng)鏡像地址。

• 部署 flannel 服務(wù)的官方下載地址

#?部署flannel服務(wù) #?1.修改鏡像地址(如果下載不了的話) #?2.修改Network為--pod-network-cidr的參數(shù)IP段 $?kubectl?apply?-f?./kube-flannel.yml#?如果部署出現(xiàn)問(wèn)題可通過(guò)如下命令查看日志 $?kubectl?logs?kube-flannel-ds-6xxs5?--namespace=kube-system $?kubectl?describe?pod?kube-flannel-ds-6xxs5?--namespace=kube-system

• 如果使用當(dāng)中存在問(wèn)題的，可以參考官方的問(wèn)題手冊(cè)

  • 因?yàn)槲覀冞@里使用的是?Vagrant?虛擬出來(lái)的機(jī)器進(jìn)行?K8S?的部署，但是在運(yùn)行對(duì)應(yīng)?yaml?配置的時(shí)候，會(huì)報(bào)錯(cuò)。通過(guò)查看日志發(fā)現(xiàn)是因?yàn)槟J(rèn)綁定的是虛擬機(jī)上面的?eth0?這塊網(wǎng)卡，而這塊網(wǎng)卡是?Vagrant?使用的，我們應(yīng)該綁定的是?eth1?才對(duì)。

  • Vagrant?通常為所有?VM?分配兩個(gè)接口，第一個(gè)為所有主機(jī)分配的?IP?地址為?10.0.2.15，用于獲得?NAT?的外部流量。這樣會(huì)導(dǎo)致?flannel?部署存在問(wèn)題。通過(guò)官方問(wèn)題說(shuō)明，我們可以使用?--iface=eth1?這個(gè)參數(shù)選擇第二個(gè)網(wǎng)卡。

  • 對(duì)應(yīng)的參數(shù)使用方式，可以參考 flannel use –iface=eth1 中的回答自行添加，而這里我直接修改了啟動(dòng)的配置文件，在啟動(dòng)服務(wù)的時(shí)候通過(guò)?args?修改了，如下所示。

$?kubectl?get?pods?-n?kube-system NAME?????????????????????????????????READY???STATUS????RESTARTS???AGE coredns-7f89b7bc75-bh42f?????????????1/1?????Running???0??????????61m coredns-7f89b7bc75-dvzpl?????????????1/1?????Running???0??????????61m etcd-k8s-master??????????????????????1/1?????Running???0??????????62m kube-apiserver-k8s-master????????????1/1?????Running???0??????????62m kube-controller-manager-k8s-master???1/1?????Running???0??????????62m kube-flannel-ds-zl148????????????????1/1?????Running???0??????????44s kube-flannel-ds-ll523????????????????1/1?????Running???0??????????44s kube-flannel-ds-wpmhw????????????????1/1?????Running???0??????????44s kube-proxy-5rlpv?????????????????????1/1?????Running???0??????????61m kube-scheduler-k8s-master????????????1/1?????Running???0??????????62m

• 配置文件如下所示

--- apiVersion:?policy/v1beta1 kind:?PodSecurityPolicy metadata:name:?psp.flannel.unprivilegedannotations:seccomp.security.alpha.kubernetes.io/allowedProfileNames:?docker/defaultseccomp.security.alpha.kubernetes.io/defaultProfileName:?docker/defaultapparmor.security.beta.kubernetes.io/allowedProfileNames:?runtime/defaultapparmor.security.beta.kubernetes.io/defaultProfileName:?runtime/default spec:privileged:?falsevolumes:-?configMap-?secret-?emptyDir-?hostPathallowedHostPaths:-?pathPrefix:?"/etc/cni/net.d"-?pathPrefix:?"/etc/kube-flannel"-?pathPrefix:?"/run/flannel"readOnlyRootFilesystem:?false#?Users?and?groupsrunAsUser:rule:?RunAsAnysupplementalGroups:rule:?RunAsAnyfsGroup:rule:?RunAsAny#?Privilege?EscalationallowPrivilegeEscalation:?falsedefaultAllowPrivilegeEscalation:?false#?CapabilitiesallowedCapabilities:?["NET_ADMIN",?"NET_RAW"]defaultAddCapabilities:?[]requiredDropCapabilities:?[]#?Host?namespaceshostPID:?falsehostIPC:?falsehostNetwork:?truehostPorts:-?min:?0max:?65535#?SELinuxseLinux:#?SELinux?is?unused?in?CaaSPrule:?"RunAsAny" --- kind:?ClusterRole apiVersion:?rbac.authorization.k8s.io/v1 metadata:name:?flannel rules:-?apiGroups:?["extensions"]resources:?["podsecuritypolicies"]verbs:?["use"]resourceNames:?["psp.flannel.unprivileged"]-?apiGroups:-?""resources:-?podsverbs:-?get-?apiGroups:-?""resources:-?nodesverbs:-?list-?watch-?apiGroups:-?""resources:-?nodes/statusverbs:-?patch --- kind:?ClusterRoleBinding apiVersion:?rbac.authorization.k8s.io/v1 metadata:name:?flannel roleRef:apiGroup:?rbac.authorization.k8s.iokind:?ClusterRolename:?flannel subjects:-?kind:?ServiceAccountname:?flannelnamespace:?kube-system --- apiVersion:?v1 kind:?ServiceAccount metadata:name:?flannelnamespace:?kube-system --- kind:?ConfigMap apiVersion:?v1 metadata:name:?kube-flannel-cfgnamespace:?kube-systemlabels:tier:?nodeapp:?flannel data:cni-conf.json:?|{"name":?"cbr0","cniVersion":?"0.3.1","plugins":?[{"type":?"flannel","delegate":?{"hairpinMode":?true,"isDefaultGateway":?true}},{"type":?"portmap","capabilities":?{"portMappings":?true}}]}net-conf.json:?|{"Network":?"10.244.0.0/16","Backend":?{"Type":?"vxlan"}} --- apiVersion:?apps/v1 kind:?DaemonSet metadata:name:?kube-flannel-dsnamespace:?kube-systemlabels:tier:?nodeapp:?flannel spec:selector:matchLabels:app:?flanneltemplate:metadata:labels:tier:?nodeapp:?flannelspec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:-?matchExpressions:-?key:?kubernetes.io/osoperator:?Invalues:-?linuxhostNetwork:?truepriorityClassName:?system-node-criticaltolerations:-?operator:?Existseffect:?NoScheduleserviceAccountName:?flannelinitContainers:-?name:?install-cniimage:?quay.io/coreos/flannel:v0.13.1-rc1command:-?cpargs:-?-f-?/etc/kube-flannel/cni-conf.json-?/etc/cni/net.d/10-flannel.conflistvolumeMounts:-?name:?cnimountPath:?/etc/cni/net.d-?name:?flannel-cfgmountPath:?/etc/kube-flannel/containers:-?name:?kube-flannelimage:?quay.io/coreos/flannel:v0.13.1-rc1command:-?/opt/bin/flanneldargs:-?--ip-masq-?--kube-subnet-mgr-?--iface=eth1resources:requests:cpu:?"100m"memory:?"50Mi"limits:cpu:?"100m"memory:?"50Mi"securityContext:privileged:?falsecapabilities:add:?["NET_ADMIN",?"NET_RAW"]env:-?name:?POD_NAMEvalueFrom:fieldRef:fieldPath:?metadata.name-?name:?POD_NAMESPACEvalueFrom:fieldRef:fieldPath:?metadata.namespacevolumeMounts:-?name:?runmountPath:?/run/flannel-?name:?flannel-cfgmountPath:?/etc/kube-flannel/volumes:-?name:?runhostPath:path:?/run/flannel-?name:?cnihostPath:path:?/etc/cni/net.d-?name:?flannel-cfgconfigMap:name:?kube-flannel-cfg

• 至此集群部署成功！如果有參數(shù)錯(cuò)誤需要修改，你也可以在?reset?后重新?init?集群。

$?kubectl?get?nodes NAME?????????STATUS???ROLES??????????????????AGE???VERSION k8s-master???Ready????control-plane,master???62m???v1.20.2 k8s-node1????Ready????control-plane,master???82m???v1.20.2 k8s-node2????Ready????control-plane,master???82m???v1.20.2#?重啟集群 $?sudo?kubeadm?reset $?sudo?kubeadm?init

4. 部署 dashboard 服務(wù)

以 WEB 頁(yè)面的可視化 dashboard 來(lái)監(jiān)控集群的狀態(tài)

這個(gè)還是會(huì)遇到需要搭梯子下載啟動(dòng)配置文件的問(wèn)題，下面是對(duì)應(yīng)的下載地址，可以下載之后上傳到服務(wù)器上面在進(jìn)行部署。

• 部署 dashboard 服務(wù)的官方下載地址

#?部署flannel服務(wù) $?kubectl?apply?-f?./kube-dashboard.yaml#?如果部署出現(xiàn)問(wèn)題可通過(guò)如下命令查看日志 $?kubectl?logs?\kubernetes-dashboard-c9fb67ffc-nknpj?\--namespace=kubernetes-dashboard $?kubectl?describe?pod?\kubernetes-dashboard-c9fb67ffc-nknpj?\--namespace=kubernetes-dashboard $?kubectl?get?svc?-n?kubernetes-dashboard NAME????????????????????????TYPE????????CLUSTER-IP??????EXTERNAL-IP???PORT(S)????AGE dashboard-metrics-scraper???ClusterIP???10.245.214.11????<none>????????8000/TCP???26s kubernetes-dashboard????????ClusterIP???10.245.161.146???<none>????????443/TCP????26s

需要注意的是?dashboard?默認(rèn)不允許外網(wǎng)訪問(wèn)，即使通過(guò)?kubectl proxy?允許外網(wǎng)訪問(wèn)。但?dashboard?又只允許?HTTPS?訪問(wèn)，這樣?kubeadm init?時(shí)自簽名的?CA?證書(shū)是不被瀏覽器承認(rèn)的。

我采用的方案是?Nginx?作為反向代理，使用?Lets Encrypt?提供的有效證書(shū)對(duì)外提供服務(wù)，再經(jīng)由?proxy_pass?指令反向代理到?kubectl proxy?上，如下所示。此時(shí)，本地可經(jīng)由?8888?訪問(wèn)到?dashboard?服務(wù)，再通過(guò)?Nginx?訪問(wèn)它。

#?代理(可以使用supervisor) $?kubectl?proxy?--accept-hosts='^*$' $?kubectl?proxy?--port=8888?--accept-hosts='^*$' #?測(cè)試代理是否正常(默認(rèn)監(jiān)聽(tīng)在8001端口上) $?curl?-X?GET?-L?http://localhost:8001#?本地(可以使用nginx) proxy_pass?http://localhost:8001; proxy_pass?http://localhost:8888;#?外網(wǎng)訪問(wèn)如下URL地址 https://mydomain/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

• 配置文件整理

  • nginx

  • supervisor

# k8s.confclient_max_body_size 80M; client_body_buffer_size 128k; proxy_connect_timeout 600; proxy_read_timeout 600; proxy_send_timeout 600;server {listen 8080 ssl;server_name _;ssl_certificate /etc/kubernetes/pki/ca.crt;ssl_certificate_key /etc/kubernetes/pki/ca.key;access_log /var/log/nginx/k8s.access.log;error_log /var/log/nginx/k8s.error.log error;location / {proxy_set_header X-Forwarded-Proto $scheme;proxy_set_header Host $http_host;proxy_set_header X-Real-IP $remote_addr;proxy_pass http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/;} } #?k8s.conf[program:k8s-master] command=kubectl?proxy?--accept-hosts='^*$' user=vagrant environment=KUBECONFIG="/home/vagrant/.kube/config" stopasgroup=true stopasgroup=true autostart=true autorestart=unexpected stdout_logfile_maxbytes=1MB stdout_logfile_backups=10 stderr_logfile_maxbytes=1MB stderr_logfile_backups=10 stderr_logfile=/var/log/supervisor/k8s-stderr.log stdout_logfile=/var/log/supervisor/k8s-stdout.log

• 配置文件如下所示

#?Copyright?2017?The?Kubernetes?Authors. # #?Licensed?under?the?Apache?License,?Version?2.0?(the?"License"); #?you?may?not?use?this?file?except?in?compliance?with?the?License. #?You?may?obtain?a?copy?of?the?License?at # #?????http://www.apache.org/licenses/LICENSE-2.0 # #?Unless?required?by?applicable?law?or?agreed?to?in?writing,?software #?distributed?under?the?License?is?distributed?on?an?"AS?IS"?BASIS, #?WITHOUT?WARRANTIES?OR?CONDITIONS?OF?ANY?KIND,?either?express?or?implied. #?See?the?License?for?the?specific?language?governing?permissions?and #?limitations?under?the?License.apiVersion:?v1 kind:?Namespace metadata:name:?kubernetes-dashboard--- apiVersion:?v1 kind:?ServiceAccount metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboardnamespace:?kubernetes-dashboard--- kind:?Service apiVersion:?v1 metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboardnamespace:?kubernetes-dashboard spec:ports:-?port:?443targetPort:?8443selector:k8s-app:?kubernetes-dashboard--- apiVersion:?v1 kind:?Secret metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboard-certsnamespace:?kubernetes-dashboard type:?Opaque--- apiVersion:?v1 kind:?Secret metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboard-csrfnamespace:?kubernetes-dashboard type:?Opaque data:csrf:?""--- apiVersion:?v1 kind:?Secret metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboard-key-holdernamespace:?kubernetes-dashboard type:?Opaque--- kind:?ConfigMap apiVersion:?v1 metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboard-settingsnamespace:?kubernetes-dashboard--- kind:?Role apiVersion:?rbac.authorization.k8s.io/v1 metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboardnamespace:?kubernetes-dashboard rules:#?Allow?Dashboard?to?get,?update?and?delete?Dashboard?exclusive?secrets.-?apiGroups:?[""]resources:?["secrets"]resourceNames:["kubernetes-dashboard-key-holder","kubernetes-dashboard-certs","kubernetes-dashboard-csrf",]verbs:?["get",?"update",?"delete"]#?Allow?Dashboard?to?get?and?update?'kubernetes-dashboard-settings'?config?map.-?apiGroups:?[""]resources:?["configmaps"]resourceNames:?["kubernetes-dashboard-settings"]verbs:?["get",?"update"]#?Allow?Dashboard?to?get?metrics.-?apiGroups:?[""]resources:?["services"]resourceNames:?["heapster",?"dashboard-metrics-scraper"]verbs:?["proxy"]-?apiGroups:?[""]resources:?["services/proxy"]resourceNames:["heapster","http:heapster:","https:heapster:","dashboard-metrics-scraper","http:dashboard-metrics-scraper",]verbs:?["get"]--- kind:?ClusterRole apiVersion:?rbac.authorization.k8s.io/v1 metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboard rules:#?Allow?Metrics?Scraper?to?get?metrics?from?the?Metrics?server-?apiGroups:?["metrics.k8s.io"]resources:?["pods",?"nodes"]verbs:?["get",?"list",?"watch"]--- apiVersion:?rbac.authorization.k8s.io/v1 kind:?RoleBinding metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboardnamespace:?kubernetes-dashboard roleRef:apiGroup:?rbac.authorization.k8s.iokind:?Rolename:?kubernetes-dashboard subjects:-?kind:?ServiceAccountname:?kubernetes-dashboardnamespace:?kubernetes-dashboard--- apiVersion:?rbac.authorization.k8s.io/v1 kind:?ClusterRoleBinding metadata:name:?kubernetes-dashboard roleRef:apiGroup:?rbac.authorization.k8s.iokind:?ClusterRolename:?kubernetes-dashboard subjects:-?kind:?ServiceAccountname:?kubernetes-dashboardnamespace:?kubernetes-dashboard--- kind:?Deployment apiVersion:?apps/v1 metadata:labels:k8s-app:?kubernetes-dashboardname:?kubernetes-dashboardnamespace:?kubernetes-dashboard spec:replicas:?1revisionHistoryLimit:?10selector:matchLabels:k8s-app:?kubernetes-dashboardtemplate:metadata:labels:k8s-app:?kubernetes-dashboardspec:containers:-?name:?kubernetes-dashboardimage:?registry.cn-shanghai.aliyuncs.com/jieee/dashboard:v2.0.4imagePullPolicy:?Alwaysports:-?containerPort:?8443protocol:?TCPargs:-?--auto-generate-certificates-?--namespace=kubernetes-dashboard#?Uncomment?the?following?line?to?manually?specify?Kubernetes?API?server?Host#?If?not?specified,?Dashboard?will?attempt?to?auto?discover?the?API?server?and?connect#?to?it.?Uncomment?only?if?the?default?does?not?work.#?-?--apiserver-host=http://my-address:portvolumeMounts:-?name:?kubernetes-dashboard-certsmountPath:?/certs#?Create?on-disk?volume?to?store?exec?logs-?mountPath:?/tmpname:?tmp-volumelivenessProbe:httpGet:scheme:?HTTPSpath:?/port:?8443initialDelaySeconds:?30timeoutSeconds:?30securityContext:allowPrivilegeEscalation:?falsereadOnlyRootFilesystem:?truerunAsUser:?1001runAsGroup:?2001volumes:-?name:?kubernetes-dashboard-certssecret:secretName:?kubernetes-dashboard-certs-?name:?tmp-volumeemptyDir:?{}serviceAccountName:?kubernetes-dashboardnodeSelector:"kubernetes.io/os":?linux#?Comment?the?following?tolerations?if?Dashboard?must?not?be?deployed?on?mastertolerations:-?key:?node-role.kubernetes.io/mastereffect:?NoSchedule--- kind:?Service apiVersion:?v1 metadata:labels:k8s-app:?dashboard-metrics-scrapername:?dashboard-metrics-scrapernamespace:?kubernetes-dashboard spec:ports:-?port:?8000targetPort:?8000selector:k8s-app:?dashboard-metrics-scraper--- kind:?Deployment apiVersion:?apps/v1 metadata:labels:k8s-app:?dashboard-metrics-scrapername:?dashboard-metrics-scrapernamespace:?kubernetes-dashboard spec:replicas:?1revisionHistoryLimit:?10selector:matchLabels:k8s-app:?dashboard-metrics-scrapertemplate:metadata:labels:k8s-app:?dashboard-metrics-scraperannotations:seccomp.security.alpha.kubernetes.io/pod:?"runtime/default"spec:containers:-?name:?dashboard-metrics-scraperimage:?registry.cn-shanghai.aliyuncs.com/jieee/metrics-scraper:v1.0.4ports:-?containerPort:?8000protocol:?TCPlivenessProbe:httpGet:scheme:?HTTPpath:?/port:?8000initialDelaySeconds:?30timeoutSeconds:?30volumeMounts:-?mountPath:?/tmpname:?tmp-volumesecurityContext:allowPrivilegeEscalation:?falsereadOnlyRootFilesystem:?truerunAsUser:?1001runAsGroup:?2001serviceAccountName:?kubernetes-dashboardnodeSelector:"kubernetes.io/os":?linux#?Comment?the?following?tolerations?if?Dashboard?must?not?be?deployed?on?mastertolerations:-?key:?node-role.kubernetes.io/mastereffect:?NoSchedulevolumes:-?name:?tmp-volumeemptyDir:?{}

• 第一種：登錄 dashboard 的方式(配置文件)

  • 采用?token?方式

  • 采用秘鑰文件方式

#?創(chuàng)建管理員帳戶(dashboard) $?cat?<<EOF?|?kubectl?apply?-f?- apiVersion:?v1 kind:?ServiceAccount metadata:name:?admin-usernamespace:?kubernetes-dashboard EOF #?將用戶綁定已經(jīng)存在的集群管理員角色 $?cat?<<EOF?|?kubectl?apply?-f?- apiVersion:?rbac.authorization.k8s.io/v1 kind:?ClusterRoleBinding metadata:name:?admin-user roleRef:apiGroup:?rbac.authorization.k8s.iokind:?ClusterRolename:?cluster-admin subjects: -?kind:?ServiceAccountname:?admin-usernamespace:?kubernetes-dashboard EOF #?獲取可用戶于訪問(wèn)的token令牌 $?kubectl?-n?kubernetes-dashboard?describe?secret?\$(kubectl?-n?kubernetes-dashboard?get?secret?\|?grep?admin-user?|?awk?'{print?$1}')

• 登錄界面展示

  • 針對(duì)?Chrome?瀏覽器，在空白處點(diǎn)擊然后輸入：thisisunsafe

  • 針對(duì)?Firefox?瀏覽器，遇到證書(shū)過(guò)期，添加例外訪問(wèn)
    ?

• 第二種：授權(quán) dashboard 權(quán)限(不適用配置文件)

  • 如果登錄之后提示權(quán)限問(wèn)題的話，可以執(zhí)行如下操作

  • 把?serviceaccount?綁定在?clusteradmin

  • 授權(quán)?serviceaccount?用戶具有整個(gè)集群的訪問(wèn)管理權(quán)限

#?創(chuàng)建serviceaccount $?kubectl?create?serviceaccount?dashboard-admin?-n?kube-system#?把serviceaccount綁定在clusteradmin #?授權(quán)serviceaccount用戶具有整個(gè)集群的訪問(wèn)管理權(quán)限 $?kubectl?create?clusterrolebinding?\dashboard-cluster-admin?--clusterrole=cluster-admin?\--serviceaccount=kube-system:dashboard-admin#?獲取serviceaccount的secret信息，可得到token令牌的信息 $?kubectl?get?secret?-n?kube-system#?通過(guò)上邊命令獲取到dashboard-admin-token-slfcr信息 $?kubectl?describe?secret?<dashboard-admin-token-slfcr>?-n?kube-system#?瀏覽器訪問(wèn)登錄并把token粘貼進(jìn)去登錄即可 https://192.168.30.30:8080/#?快捷查看token的命令 $?kubectl?describe?secrets?-n?kube-system?\$(kubectl?-n?kube-system?get?secret?|?awk?'/admin/{print?$1}')

總結(jié)

以上是生活随笔為你收集整理的手把手教你部署一个最小化的 Kubernetes 集群的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。