1 使用statefullset部署mysql主從集群

mysql-0是master
mysql-1和mysql-2是兩個備份
當(dāng)mysql寫的時候，找headless service中的mysql-0.mysql；當(dāng)mysql讀的時候，找clusterip service中的mysql讀，實現(xiàn)讀寫分離。
statefulset維護(hù)三個副本，一個主兩個從
因為主從的配置不同所以通過configmap來解決主從配置不同的問題
部署mysql文檔

1.1配置configmap(創(chuàng)建mysql配置）

[root@server2 statefulset]# mkdir mysql [root@server2 statefulset]# cd mysql/ [root@server2 mysql]# \vi cm.yaml 將官網(wǎng)的例子復(fù)制過來 apiVersion: v1 kind: ConfigMap metadata:name: mysqllabels:app: mysql data:master.cnf: |[mysqld]log-bin # 主mysql激活二進(jìn)制日志slave.cnf: |[mysqld]super-read-only # 從mysql上面設(shè)置為只讀 [root@server2 mysql]# kubectl apply -f cm.yaml 運行 configmap/mysql created [root@server2 mysql]# kubectl get cm NAME DATA AGE kube-root-ca.crt 1 7h40m mysql 2 13m

1.2創(chuàng)建service服務(wù)

[root@server2 mysql]# \vi svc.yaml 將官網(wǎng)的例子復(fù)制過來，然后用vim打開 [root@server2 mysql]# vim svc.yaml 創(chuàng)建兩個svc，主要用于讀寫分離 apiVersion: v1 kind: Service metadata:name: mysqllabels:app: mysql spec:ports:- name: mysql 負(fù)責(zé)寫port: 3306clusterIP: Noneselector:app: mysql --- apiVersion: v1 kind: Service metadata:name: mysql-read 只讀labels:app: mysql spec:ports:- name: mysqlport: 3306selector:app: mysql [root@server2 mysql]# kubectl apply -f svc.yaml 創(chuàng)建 service/mysql created service/mysql-read created [root@server2 mysql]# kubectl get svc 查看svc，創(chuàng)建成功 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h39m mysql ClusterIP None <none> 3306/TCP 28s mysql-read ClusterIP 10.101.37.222 <none> 3306/TCP 28s

1.3 statefulset配置

[root@server1 ~]# docker pull mysql:5.7 拉取mysql ，注意版本必須是5.7 [root@foundation50 k8s]# scp mysql-xtrabackup.tar server1: [root@server1 ~]# docker load -i mysql-xtrabackup.tar 導(dǎo)入鏡像 [root@server1 harbor]# docker push reg.westos.org/library/mysql:5.7 上傳鏡像到倉庫 [root@server1 harbor]# docker push reg.westos.org/library/xtrabackup:1.0 上傳鏡像到倉庫 [root@server2 mysql]# vim deploy.yaml apiVersion: apps/v1 kind: StatefulSet metadata:name: mysql spec:selector:matchLabels:app: mysqlserviceName: mysqlreplicas: 3template:metadata:labels:app: mysqlspec:initContainers:- name: init-mysqlimage: mysql:5.7command:- bash- "-c"- |set -ex# Generate mysql server-id from pod ordinal index.[[ `hostname` =~ -([0-9]+)$ ]] || exit 1 過濾第一個pod名mysql，其中hostname表示容器內(nèi)pod名字ordinal=${BASH_REMATCH[1]} 表示拿取第一個數(shù)字0echo [mysqld] > /mnt/conf.d/server-id.cnf 創(chuàng)建一個server-id.cnf 文件，往里面寫mysqld表示這個文件里的配置針對server生效# Add an offset to avoid reserved server-id=0 value.echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf 往server-id.conf 追加一條server-id=100+上面取到的數(shù)字# Copy appropriate conf.d files from config-map to emptyDir. 拷貝conf.d 文件到空卷if [[ $ordinal -eq 0 ]]; then 如果取到的數(shù)字為0cp /mnt/config-map/master.cnf /mnt/conf.d/ 拷貝masterelsecp /mnt/config-map/slave.cnf /mnt/conf.d/ 否則拷貝slavefi volumeMounts:- name: confmountPath: /mnt/conf.d- name: config-mapmountPath: /mnt/config-mapcontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ALLOW_EMPTY_PASSWORDvalue: "1"ports:- name: mysqlcontainerPort: 3306volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql 掛在點里創(chuàng)建一個子目錄- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 500mmemory: 1GilivenessProbe:exec:command: ["mysqladmin", "ping"]initialDelaySeconds: 30periodSeconds: 10timeoutSeconds: 5readinessProbe:exec:# Check we can execute queries over TCP (skip-networking is off).command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]initialDelaySeconds: 5periodSeconds: 2timeoutSeconds: 1volumes:- name: confemptyDir: {}- name: config-mapconfigMap:name: mysqlvolumeClaimTemplates: - metadata:name: dataspec:accessModes: ["ReadWriteOnce"]resources:requests:storage: 10Gi [root@server2 mysql]# kubectl apply -f deploy.yaml 創(chuàng)建 [root@server2 mysql]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-0 1/1 Running 0 71s [root@server2 mysql]# kubectl get pv pv已經(jīng)生成 NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-179b1b5e-3d25-4741-8458-a99b18f175de 1Gi RWX Delete Bound default/test-claim managed-nfs-storage 5h6m pvc-4b2ad3b8-d156-429a-bc7a-f6616534e14f 10Gi RWO Delete Bound default/data-mysql-0 managed-nfs-storage 5h [root@server2 mysql]# kubectl get pvc pvc已經(jīng)生成 NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE data-mysql-0 Bound pvc-4b2ad3b8-d156-429a-bc7a-f6616534e14f 10Gi RWO managed-nfs-storage 5h test-claim Bound pvc-179b1b5e-3d25-4741-8458-a99b18f175de 1Gi RWX managed-nfs-storage 5h6m [root@server1 ~]# cd /nfsdata/ 查看共享目錄 [root@server1 nfsdata]# ls default-data-mysql-0-pvc-4b2ad3b8-d156-429a-bc7a-f6616534e14f [root@server1 nfsdata]# cd default-data-mysql-0-pvc-4b2ad3b8-d156-429a-bc7a-f6616534e14f/ [root@server1 default-data-mysql-0-pvc-4b2ad3b8-d156-429a-bc7a-f6616534e14f]# ls mysql 創(chuàng)建的子目錄 [root@server1 mysql]# ls auto.cnf ibdata1 mysql-0-bin.000002 public_key.pem ca-key.pem ib_logfile0 mysql-0-bin.000003 server-cert.pem ca.pem ib_logfile1 mysql-0-bin.000004 server-key.pem client-cert.pem ibtmp1 mysql-0-bin.index sys client-key.pem mysql performance_schema ib_buffer_pool mysql-0-bin.000001 private_key.pem [root@server2 mysql]# kubectl logs mysql-0 -c init-mysql 查看初始化容器日志輸出 ++ hostname + [[ mysql-0 =~ -([0-9]+)$ ]] + ordinal=0 + echo '[mysqld]' + echo server-id=100 + [[ 0 -eq 0 ]] + cp /mnt/config-map/master.cnf /mnt/conf.d/ [root@server2 mysql]# kubectl exec mysql-0 -c mysql -- ls /etc/mysql/conf.d 查看容器掛載目錄的內(nèi)容，有兩個文件 master.cnf server-id.cnf [root@server2 mysql]# kubectl exec mysql-0 -c mysql -- cat /etc/mysql/conf.d/master.cnf 查看文件具體內(nèi)容 [mysqld] log-bin [root@server2 mysql]# kubectl exec mysql-0 -c mysql -- cat /etc/mysql/conf.d/server-id.cnf 查看文件具體內(nèi)容 [mysqld] server-id=100 上述master已經(jīng)設(shè)置成功，該設(shè)置slave了

[root@server2 mysql]# kubectl apply -f deploy.yaml 創(chuàng)建，刪除pod statefulset.apps/mysql configured [root@server2 mysql]# vim deploy.yaml apiVersion: apps/v1 kind: StatefulSet metadata:name: mysql spec:selector:matchLabels:app: mysqlserviceName: mysqlreplicas: 1 1個副本數(shù)template:metadata:labels:app: mysqlspec:initContainers:- name: init-mysqlimage: mysql:5.7command:- bash- "-c"- |set -ex# Generate mysql server-id from pod ordinal index.[[ `hostname` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}echo [mysqld] > /mnt/conf.d/server-id.cnf# Add an offset to avoid reserved server-id=0 value.echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf# Copy appropriate conf.d files from config-map to emptyDir.if [[ $ordinal -eq 0 ]]; thencp /mnt/config-map/master.cnf /mnt/conf.d/elsecp /mnt/config-map/slave.cnf /mnt/conf.d/fi volumeMounts:- name: confmountPath: /mnt/conf.d- name: config-mapmountPath: /mnt/config-mapmountPath: /mnt/config-map- name: clone-mysql 在科隆一個鏡像xtrabackup，用于接收mysql-0的數(shù)據(jù)image: xtrabackup:1.0command:- bash- "-c"- |set -ex# Skip the clone if data already exists.[[ -d /var/lib/mysql/mysql ]] && exit 0# Skip the clone on master (ordinal index 0).[[ `hostname` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}[[ $ordinal -eq 0 ]] && exit 0# Clone data from previous peer.ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql 接收數(shù)據(jù)# Prepare the backup.xtrabackup --prepare --target-dir=/var/lib/mysql volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dcontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ALLOW_EMPTY_PASSWORDvalue: "1"ports:- name: mysqlcontainerPort: 3306volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 500mmemory: 1GilivenessProbe:exec:command: ["mysqladmin", "ping"]initialDelaySeconds: 30periodSeconds: 10timeoutSeconds: 5readinessProbe:exec:# Check we can execute queries over TCP (skip-networking is off).command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]initialDelaySeconds: 5periodSeconds: 2timeoutSeconds: 1- name: xtrabackup image: xtrabackup:1.0ports:- name: xtrabackupcontainerPort: 3307command:- bash- "-c"- |set -excd /var/lib/mysql# Determine binlog position of cloned data, if any.if [[ -f xtrabackup_slave_info && "x$(<xtrabackup_slave_info)" != "x" ]]; then# XtraBackup already generated a partial "CHANGE MASTER TO" query# because we're cloning from an existing slave. (Need to remove the tailing semicolon!)cat xtrabackup_slave_info | sed -E 's/;$//g' > change_master_to.sql.in# Ignore xtrabackup_binlog_info in this case (it's useless).rm -f xtrabackup_slave_info xtrabackup_binlog_infoelif [[ -f xtrabackup_binlog_info ]]; then# We're cloning directly from master. Parse binlog position.[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1rm -f xtrabackup_binlog_info xtrabackup_slave_infoecho "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.infi# Check if we need to complete a clone by starting replication.if [[ -f change_master_to.sql.in ]]; thenecho "Waiting for mysqld to be ready (accepting connections)"until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; doneecho "Initializing replication from clone position"mysql -h 127.0.0.1 \-e "$(<change_master_to.sql.in), \MASTER_HOST='mysql-0.mysql', \MASTER_USER='root', \MASTER_PASSWORD='', \MASTER_CONNECT_RETRY=10; \START SLAVE;" || exit 1# In case of container restart, attempt this at-most-once.mv change_master_to.sql.in change_master_to.sql.origfi# Start a server to send backups when requested by peers.exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \ 發(fā)送數(shù)據(jù)"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root" volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 100mmemory: 100Mivolumes:- name: confemptyDir: {}- name: config-mapconfigMap:name: mysqlvolumeClaimTemplates:- metadata:name: dataspec:accessModes: ["ReadWriteOnce"]resources:requests:storage: 10Gi [root@server2 mysql]# kubectl apply -f deploy.yaml 創(chuàng)建 statefulset.apps/mysql configured [root@server2 mysql]# kubectl get pod 容器運行成功 NAME READY STATUS RESTARTS AGE mysql-0 2/2 Running 0 3m41s [root@server2 mysql]# kubectl logs mysql-0 -c xtrabackup 查看日志輸出 + cd /var/lib/mysql + [[ -f xtrabackup_slave_info ]] + [[ -f xtrabackup_binlog_info ]] + [[ -f change_master_to.sql.in ]] + exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c 'xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root' [root@server2 mysql]# kubectl logs mysql-0 -c clone-mysql + [[ -d /var/lib/mysql/mysql ]] + exit 0 [root@server2 mysql]# vim deploy.yaml

[root@server2 mysql]# kubectl logs mysql-1 -c init-mysql 查看初始化日志輸出 ++ hostname + [[ mysql-1 =~ -([0-9]+)$ ]] + ordinal=1 數(shù)字取1 + echo '[mysqld]' + echo server-id=101 server-id 就是100+1 + [[ 1 -eq 0 ]] id不等0則拷貝slave.cnf + cp /mnt/config-map/slave.cnf /mnt/conf.d/ [root@server2 mysql]# kubectl logs mysql-1 clone-mysql 查看clone日志輸出 + [[ -d /var/lib/mysql/mysql ]] ++ hostname + [[ mysql-1 =~ -([0-9]+)$ ]] + ordinal=1 + [[ 1 -eq 0 ]] + ncat --recv-only mysql-0.mysql 3307 開啟一個3307端口，接收mysql-0數(shù)據(jù) + xbstream -x -C /var/lib/mysql + xtrabackup --prepare --target-dir=/var/lib/mysql

查看克隆日志輸出

[root@server2 mysql]# vim deploy.yaml

[root@server2 mysql]# kubectl apply -f deploy.yaml 創(chuàng)建 [root@server2 mysql]# kubectl get pod 創(chuàng)建成功 NAME READY STATUS RESTARTS AGE mysql-0 2/2 Terminating 0 37m mysql-1 2/2 Running 0 101s mysql-2 2/2 Running 1 (22s ago) 89s [root@server2 mysql]# kubectl logs mysql-2 clone-mysql + [[ -d /var/lib/mysql/mysql ]] ++ hostname + [[ mysql-2 =~ -([0-9]+)$ ]] + ordinal=2 + [[ 2 -eq 0 ]] + ncat --recv-only mysql-1.mysql 3307 第2個slave是從第一個slave拿數(shù)據(jù) + xbstream -x -C /var/lib/mysql + xtrabackup --prepare --target-dir=/var/lib/mysql [root@server2 mysql]# kubectl logs mysql-2 xtrabackup + cd /var/lib/mysql + [[ -f xtrabackup_slave_info ]] 直接獲取上一個slave標(biāo)志，這里面已經(jīng)有之前的sql語句 + [[ xCHANGE MASTER TO MASTER_LOG_FILE='mysql-0-bin.000005', MASTER_LOG_POS=154 != \x ]] + cat xtrabackup_slave_info + sed -E 's/;$//g' + rm -f xtrabackup_slave_info xtrabackup_binlog_info + [[ -f change_master_to.sql.in ]] Initializing replication from clone position + mysql -h 127.0.0.1 -e 'CHANGE MASTER TO MASTER_LOG_FILE='\''mysql-0-bin.000005'\'', MASTER_LOG_POS=154, MASTER_HOST='\''mysql-0.mysql'\'', MASTER_USER='\''root'\'', MASTER_PASSWORD='\'''\'', MASTER_CONNECT_RETRY=10; START SLAVE;' + mv change_master_to.sql.in change_master_to.sql.orig + exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c 'xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root'

測試：

總結(jié)

以上是生活随笔為你收集整理的k8s（七）—statefullset部署mysql主从集群的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。