摘要：接觸docker以來(lái)，似乎養(yǎng)成了一種習(xí)慣，安裝什么應(yīng)用軟件都想往docker方向做，今天就想來(lái)嘗試下使用docker搭建redis集群。

首先，我們需要理論知識(shí)：Redis Cluster是Redis的分布式解決方案，它解決了redis單機(jī)中心化的問(wèn)題，分布式數(shù)據(jù)庫(kù)——首要解決把整個(gè)數(shù)據(jù)集按照分區(qū)規(guī)則映射到多個(gè)節(jié)點(diǎn)的問(wèn)題。

這邊就需要知道分區(qū)規(guī)則——哈希分區(qū)規(guī)則。Redis Cluster?采用哈希分區(qū)規(guī)則中的虛擬槽分區(qū)。所有的鍵根據(jù)哈希函數(shù)映射到0 ～ 16383，計(jì)算公式：slot = CRC16(key)&16383。每一個(gè)節(jié)點(diǎn)負(fù)責(zé)維護(hù)一部分槽以及槽所映射的鍵值數(shù)據(jù)。

一、創(chuàng)建redis docker基礎(chǔ)鏡像

1. 下載redis安裝包，使用版本為：4.0.1

   1 2 3

   [root@etcd1 tmp]# mkdir docker_redis_cluster [root@etcd1 tmp]# cd docker_redis_cluster/ [root@etcd2 docker_redis_cluster]# wget http://download.redis.io/releases/redis-4.0.1.tar.gz

2. 解壓編譯redis

   1 2 3

   [root@etcd1 docker_redis_cluster]# tar zxvf redis-4.0.1.tar.gz [root@etcd1 docker_redis_cluster]# cd redis-4.0.1/ [root@etcd1 redis-4.0.1]# make

3. 修改redis配置

   1	[root@etcd3 redis-4.0.1]# vi /tmp/docker_redis_cluster/redis-4.0.1/redis.conf

   　　
   修改bind ip地址

   1 2 3 4 5 6 7 8 9 10 11 12 13

   # ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the # internet, binding to all the interfaces is dangerous and will expose the # instance to everybody on the internet. So by default we uncomment the # following bind directive, that will force Redis to listen only into # the IPv4 lookback interface address (this means Redis will be able to # accept connections only from clients running into the same computer it # is running). # # IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES # JUST COMMENT THE FOLLOWING LINE. # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #bind 127.0.0.1 bind 0.0.0.0

   　　

   將守護(hù)進(jìn)程yes改成no

   1 2 3

   # By default Redis does not run as a daemon. Use 'yes' if you need it. # Note that Redis will write a pid file in /var/run/redis.pid when daemonized. daemonize no

   　　
   將密碼項(xiàng)注釋去掉，添加新密碼

   1 2 3 4 5

   # Warning: since Redis is pretty fast an outside user can try up to # 150k passwords per second against a good box. This means that you should # use a very strong password otherwise it will be very easy to break. # # requirepass foobared

   　　修改為

   1 2 3 4 5

   # Warning: since Redis is pretty fast an outside user can try up to # 150k passwords per second against a good box. This means that you should # use a very strong password otherwise it will be very easy to break. # requirepass 123456

   
   因?yàn)榕渲昧嗣艽a，所以，配置中另外一處主從連接也需要配置密碼

   1 2 3 4 5 6

   # If the master is password protected (using the "requirepass" configuration # directive below) it is possible to tell the slave to authenticate before # starting the replication synchronization process, otherwise the master will # refuse the slave request. # # masterauth <master-password>

   　　修改為

   1 2 3 4 5 6 7

   # If the master is password protected (using the "requirepass" configuration # directive below) it is possible to tell the slave to authenticate before # starting the replication synchronization process, otherwise the master will # refuse the slave request. # # masterauth <master-password> masterauth 123456

   　　

   　　
   
   設(shè)置日志路徑

   1 2 3 4

   # Specify the log file name. Also the empty string can be used to force # Redis to log on the standard output. Note that if you use standard # output for logging but daemonize, logs will be sent to /dev/null logfile?"/var/log/redis/redis-server.log"

   　　
   配置集群相關(guān)信息，去掉配置項(xiàng)前面的注釋

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

   # Normal Redis instances can't be part of a Redis Cluster; only nodes that are # started as cluster nodes can. In order to start a Redis instance as a # cluster node enable the cluster support uncommenting the following: # cluster-enabled?yes ? # Every cluster node has a cluster configuration file. This file is not # intended to be edited by hand. It is created and updated by Redis nodes. # Every Redis Cluster node requires a different cluster configuration file. # Make sure that instances running in the same system do not have # overlapping cluster configuration file names. # cluster-config-file?nodes-6379.conf ? # Cluster node timeout is the amount of milliseconds a node must be unreachable # for it to be considered in failure state. # Most other internal time limits are multiple of the node timeout. # cluster-node-timeout 15000

4. 鏡像制作

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

   [root@etcd3 docker_redis_cluster]# cd /tmp/docker_redis_cluster [root@etcd3 docker_redis_cluster]# vi Dockerfile # Redis # Version 4.0.1 ? FROM Centos:7<br> ENV REDIS_HOME?/usr/local<br> ADD redis-4.0.1.tar.gz /?# 本地的redis源碼包復(fù)制到鏡像的根路徑下，ADD命令會(huì)在復(fù)制過(guò)后自動(dòng)解包。被復(fù)制的對(duì)象必須處于Dockerfile同一路徑，且ADD后面必須使用相對(duì)路徑 RUN?mkdir?-p $REDIS_HOME/redis?# 創(chuàng)建安裝目錄 ADD redis-4.0.1/redis.conf $REDIS_HOME/redis/??# 將一開(kāi)始編譯產(chǎn)生并修改后的配置復(fù)制到安裝目錄 ? RUN yum -y update??# 更新yum源 RUN yum?install?-y?gcc?make?# 安裝編譯需要的工具 ? WORKDIR?/redis-4.0.1 RUN?make RUN?mv?/redis-4.0.1/src/redis-server??$REDIS_HOME/redis/???# 編譯后，容器中只需要可執(zhí)行文件redis-server ? WORKDIR / RUN?rm?-rf?/redis-4.0.1??????????# 刪除解壓文件 ? RUN yum remove -y?gcc?make???# 安裝編譯完成之后，可以刪除多余的gcc跟make ? VOLUME ["/var/log/redis"]??# 添加數(shù)據(jù)卷 ? EXPOSE 6379???# 暴露6379端口，也可以暴露多個(gè)端口，這里不需要如此

   　　
   PS.當(dāng)前鏡像非可執(zhí)行鏡像，所以沒(méi)有包含ENTRYPOINT和CMD指令

5. 構(gòu)建鏡像

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

   # 切換中國(guó)源 [root@etcd3 docker_redis_cluster]# vi /etc/docker/daemon.json { ??"registry-mirrors": ["https://registry.docker-cn.com"] } ? # 編譯 [root@etcd3 docker_redis_cluster]# docker build -t hakimdstx/cluster-redis . ... ? Complete! ?---> 546cb1d34f35 Removing intermediate container 6b6556c5f28d Step 14/15?: VOLUME?/var/log/redis ?---> Running?in?05a6642e4046 ?---> e7e2fb8676b2 Removing intermediate container 05a6642e4046 Step 15/15?: EXPOSE 6379 ?---> Running?in?5d7abe1709e2 ?---> 2d1322475f79 Removing intermediate container 5d7abe1709e2 Successfully built 2d1322475f79

   　　
   鏡像制作完成，制作中間可能會(huì)報(bào)：?Public key?for?glibc-headers-2.17-222.el7.x86_64.rpm?is?not installed?錯(cuò)誤，這時(shí)候需要在鏡像配置中添加一句命令：

   1 2 3 4

   ... RUN rpm --import?/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 RUN yum -y update??# 更新yum源 RUN yum?install?-y?gcc?make?# 安裝編譯需要的工具

   
   查看鏡像：

   1 2 3 4

   [root@etcd3 docker_redis_cluster]# docker images REPOSITORY????????????????????????????????? TAG???????????????? IMAGE ID??????????? CREATED???????????? SIZE hakimdstx/cluster-redis?????????????????????4.0.1?????????????? 1fca5a08a4c7??????? 14 seconds ago????? 435 MB centos????????????????????????????????????? 7?????????????????? 49f7960eb7e4??????? 2 days ago????????? 200 MB

   　

   以上，redis 基礎(chǔ)鏡像就制作完成了

二、制作redis節(jié)點(diǎn)鏡像

1. 基于此前制作的redis基礎(chǔ)鏡像創(chuàng)建一個(gè)redis節(jié)點(diǎn)鏡像

   1 2 3 4 5 6 7 8 9 10 11

   [root@etcd3 tmp]# mkdir docker_redis_nodes [root@etcd3 tmp]# cd docker_redis_nodes [root@etcd3 docker_redis_nodes]# vi Dockerfile # Redis Node # Version 4.0.1<br> FROM hakimdstx/cluster-redis:4.0.1 ? # MAINTAINER_INFO MAINTAINER hakim 1194842583@qq.com ? ENTRYPOINT ["/usr/local/redis/redis-server",?"/usr/local/redis/redis.conf"]

   　　

2. 構(gòu)建redis節(jié)點(diǎn)鏡像

   1 2 3 4 5 6 7 8 9 10 11 12 13

   [root@etcd3 docker_redis_nodes]# docker build -t hakimdstx/nodes-redis:4.0.1 .?????? Sending build context to Docker daemon 2.048 kB Step 1/3?: FROM hakimdstx/cluster-redis:4.0.1 ?---> 1fca5a08a4c7 Step 2/3?: MAINTAINER hakim 1194842583@qq.com ?---> Running?in?cc6e07eb2c36 ?---> 55769d3bfacb Removing intermediate container cc6e07eb2c36 Step 3/3?: ENTRYPOINT?/usr/local/redis/redis-server?/usr/local/redis/redis.conf ?---> Running?in?f5dedf88f6f6 ?---> da64da483559 Removing intermediate container f5dedf88f6f6 Successfully built da64da483559

   　　

3. 查看鏡像

   1 2 3 4 5

   [root@etcd3 docker_redis_nodes]# docker images REPOSITORY????????????????????????????????? TAG???????????????? IMAGE ID??????????? CREATED???????????? SIZE hakimdstx/nodes-redis???????????????????????4.0.1?????????????? da64da483559??????? 51 seconds ago????? 435 MB hakimdstx/cluster-redis?????????????????????4.0.1?????????????? 1fca5a08a4c7??????? 9 minutes ago?????? 435 MB centos????????????????????????????????????? 7?????????????????? 49f7960eb7e4??????? 2 days ago????????? 200 MB

   　　

三、運(yùn)行redis集群

1. 運(yùn)行redis容器

   1 2 3 4 5 6 7 8 9 10 11 12

   [root@etcd3 docker_redis_nodes]# docker run -d --name redis-6379 -p 6379:6379 hakimdstx/nodes-redis:4.0.1?? 1673a7d859ea83257d5bf14d82ebf717fb31405c185ce96a05f597d8f855aa7d [root@etcd3 docker_redis_nodes]# docker run -d --name redis-6380 -p 6380:6379 hakimdstx/nodes-redis:4.0.1??? df6ebce6f12a6f3620d5a29adcfbfa7024e906c3af48f21fa7e1fa524a361362 [root@etcd3 docker_redis_nodes]# docker run -d --name redis-6381 -p 6381:6379 hakimdstx/nodes-redis:4.0.1?? 396e174a1d9235228b3c5f0266785a12fb1ea49efc7ac755c9e7590e17aa1a79 [root@etcd3 docker_redis_nodes]# docker run -d --name redis-6382 -p 6382:6379 hakimdstx/nodes-redis:4.0.1 d9a71dd3f969094205ffa7596c4a04255575cdd3acca2d47fe8ef7171a3be528 [root@etcd3 docker_redis_nodes]# docker run -d --name redis-6383 -p 6383:6379 hakimdstx/nodes-redis:4.0.1 73e4f843d8cb28595456e21b04f97d18ce1cdf8dc56d1150844ba258a3781933 [root@etcd3 docker_redis_nodes]# docker run -d --name redis-6384 -p 6384:6379 hakimdstx/nodes-redis:4.0.1 10c62aafa4dac47220daf5bf3cec84406f086d5261599b54ec6c56bb7da97d6d

2. 查看容器信息

   1 2 3 4 5 6 7 8

   [root@etcd3 redis]# docker ps CONTAINER ID??????? IMAGE???????????????????????? COMMAND????????????????? CREATED???????????? STATUS????????????? PORTS??????????????????? NAMES 10c62aafa4da??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 seconds ago?????? Up 2 seconds??????? 0.0.0.0:6384->6379/tcp???redis-6384 73e4f843d8cb??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???12 seconds ago????? Up 10 seconds?????? 0.0.0.0:6383->6379/tcp???redis-6383 d9a71dd3f969??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???20 seconds ago????? Up 18 seconds?????? 0.0.0.0:6382->6379/tcp???redis-6382 396e174a1d92??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 days ago????????? Up 3 days?????????? 0.0.0.0:6381->6379/tcp???redis-6381 df6ebce6f12a??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 days ago????????? Up 3 days?????????? 0.0.0.0:6380->6379/tcp???redis-6380 1673a7d859ea??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 days ago????????? Up 3 days?????????? 0.0.0.0:6379->6379/tcp???redis-6379

   　　

3. 運(yùn)行 redis 集群容器
   1. 通過(guò)遠(yuǎn)程連接，查看redis ?info replication?信息

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

      [root@etcd2 ~]#? redis-cli -h 192.168.10.52 -p 6379 192.168.10.52:6379> info replication NOAUTH Authentication required. 192.168.10.52:6379> auth 123456 OK 192.168.10.52:6379> info replication # Replication role:master connected_slaves:0 master_replid:2f0a7b50aed699fa50a79f3f7f9751a070c50ee9 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0 192.168.10.52:6379> # 其余基本信息同上

      　　可以看到，客戶(hù)連接之后，因?yàn)橹霸O(shè)置了密碼，所以需要先輸入密碼認(rèn)證，否則就無(wú)法通過(guò)。以上信息，我們知道所有的redis都是master角色?role:master?，這顯然不是我們所希望的。

   2. 在配置之前我們需要查看所有容器當(dāng)前的IP地址

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

      [root@etcd3 redis]# docker ps CONTAINER ID??????? IMAGE???????????????????????? COMMAND????????????????? CREATED???????????? STATUS????????????? PORTS??????????????????? NAMES 10c62aafa4da??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 seconds ago?????? Up 2 seconds??????? 0.0.0.0:6384->6379/tcp???redis-6384 73e4f843d8cb??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???12 seconds ago????? Up 10 seconds?????? 0.0.0.0:6383->6379/tcp???redis-6383 d9a71dd3f969??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???20 seconds ago????? Up 18 seconds?????? 0.0.0.0:6382->6379/tcp???redis-6382 396e174a1d92??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 days ago????????? Up 3 days?????????? 0.0.0.0:6381->6379/tcp???redis-6381 df6ebce6f12a??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 days ago????????? Up 3 days?????????? 0.0.0.0:6380->6379/tcp???redis-6380 1673a7d859ea??????? hakimdstx/nodes-redis:4.0.1???"/usr/local/redis/..."???3 days ago????????? Up 3 days?????????? 0.0.0.0:6379->6379/tcp???redis-6379 [root@etcd3 redis]# [root@etcd3 redis]# docker inspect 10c62aafa4da 73e4f843d8cb d9a71dd3f969 396e174a1d92 df6ebce6f12a 1673a7d859ea | grep IPA ????????????"SecondaryIPAddresses": null, ????????????"IPAddress":?"172.17.0.7", ????????????????????"IPAMConfig": null, ????????????????????"IPAddress":?"172.17.0.7", ????????????"SecondaryIPAddresses": null, ????????????"IPAddress":?"172.17.0.6", ????????????????????"IPAMConfig": null, ????????????????????"IPAddress":?"172.17.0.6", ????????????"SecondaryIPAddresses": null, ????????????"IPAddress":?"172.17.0.5", ????????????????????"IPAMConfig": null, ????????????????????"IPAddress":?"172.17.0.5", ????????????"SecondaryIPAddresses": null, ????????????"IPAddress":?"172.17.0.4", ????????????????????"IPAMConfig": null, ????????????????????"IPAddress":?"172.17.0.4", ????????????"SecondaryIPAddresses": null, ????????????"IPAddress":?"172.17.0.3", ????????????????????"IPAMConfig": null, ????????????????????"IPAddress":?"172.17.0.3", ????????????"SecondaryIPAddresses": null, ????????????"IPAddress":?"172.17.0.2", ????????????????????"IPAMConfig": null, ????????????????????"IPAddress":?"172.17.0.2",

      ?

      可以知道：??redis-6379：172.17.0.2，redis-6380：172.17.0.3，redis-6381：172.17.0.4，redis-6382：172.17.0.5，redis-6383：172.17.0.6，redis-6384：172.17.0.7??

   3. 配置redis
   4. ert
4. Redis Cluster 的集群感知操作

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

   //集群(cluster)? CLUSTER INFO 打印集群的信息? CLUSTER NODES 列出集群當(dāng)前已知的所有節(jié)點(diǎn)（node），以及這些節(jié)點(diǎn)的相關(guān)信息。?? ??? //節(jié)點(diǎn)(node)? CLUSTER MEET <ip> <port> 將 ip 和 port 所指定的節(jié)點(diǎn)添加到集群當(dāng)中，讓它成為集群的一份子。? CLUSTER FORGET <node_id> 從集群中移除 node_id 指定的節(jié)點(diǎn)。? CLUSTER REPLICATE <node_id> 將當(dāng)前節(jié)點(diǎn)設(shè)置為 node_id 指定的節(jié)點(diǎn)的從節(jié)點(diǎn)。? CLUSTER SAVECONFIG 將節(jié)點(diǎn)的配置文件保存到硬盤(pán)里面。?? ??? //槽(slot)? CLUSTER ADDSLOTS <slot> [slot ...] 將一個(gè)或多個(gè)槽（slot）指派（assign）給當(dāng)前節(jié)點(diǎn)。? CLUSTER DELSLOTS <slot> [slot ...] 移除一個(gè)或多個(gè)槽對(duì)當(dāng)前節(jié)點(diǎn)的指派。? CLUSTER FLUSHSLOTS 移除指派給當(dāng)前節(jié)點(diǎn)的所有槽，讓當(dāng)前節(jié)點(diǎn)變成一個(gè)沒(méi)有指派任何槽的節(jié)點(diǎn)。? CLUSTER SETSLOT <slot> NODE <node_id> 將槽 slot 指派給 node_id 指定的節(jié)點(diǎn)，如果槽已經(jīng)指派給另一個(gè)節(jié)點(diǎn)，那么先讓另一個(gè)節(jié)點(diǎn)刪除該槽>，然后再進(jìn)行指派。? CLUSTER SETSLOT <slot> MIGRATING <node_id> 將本節(jié)點(diǎn)的槽 slot 遷移到 node_id 指定的節(jié)點(diǎn)中。? CLUSTER SETSLOT <slot> IMPORTING <node_id> 從 node_id 指定的節(jié)點(diǎn)中導(dǎo)入槽 slot 到本節(jié)點(diǎn)。? CLUSTER SETSLOT <slot> STABLE 取消對(duì)槽 slot 的導(dǎo)入（import）或者遷移（migrate）。?? ??? //鍵 (key)? CLUSTER KEYSLOT <key> 計(jì)算鍵 key 應(yīng)該被放置在哪個(gè)槽上。? CLUSTER COUNTKEYSINSLOT <slot> 返回槽 slot 目前包含的鍵值對(duì)數(shù)量。? CLUSTER GETKEYSINSLOT <slot> <count> 返回 count 個(gè) slot 槽中的鍵。?

   　　
   redis 集群感知：節(jié)點(diǎn)握手——是指一批運(yùn)行在集群模式的節(jié)點(diǎn)通過(guò)Gossip協(xié)議彼此通信，達(dá)到感知對(duì)方的過(guò)程。
   ?

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

   192.168.10.52:6379> CLUSTER MEET 172.17.0.3 6379 OK 192.168.10.52:6379> CLUSTER MEET 172.17.0.4 6379 OK 192.168.10.52:6379> CLUSTER MEET 172.17.0.5 6379 OK 192.168.10.52:6379> CLUSTER MEET 172.17.0.6 6379 OK 192.168.10.52:6379> CLUSTER MEET 172.17.0.7 6379 OK 192.168.10.52:6379>? CLUSTER NODES 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 172.17.0.3:6379@16379 master - 0 1528697195600 1 connected f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 172.17.0.4:6379@16379 master - 0 1528697195600 0 connected ae86224a3bc29c4854719c83979cb7506f37787a 172.17.0.7:6379@16379 master - 0 1528697195600 5 connected 98aebcfe42d8aaa8a3375e4a16707107dc9da683 172.17.0.6:6379@16379 master - 0 1528697194000 4 connected 0bbdc4176884ef0e3bb9b2e7d03d91b0e7e11f44 172.17.0.5:6379@16379 master - 0 1528697194995 3 connected 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 172.17.0.2:6379@16379 myself,master - 0 1528697195000 2 connected

   　　
   當(dāng)前已經(jīng)使這六個(gè)節(jié)點(diǎn)組成集群，但是現(xiàn)在還無(wú)法工作，因?yàn)榧汗?jié)點(diǎn)還沒(méi)有分配槽（slot）。

   1. 分配槽信息
      查看172.17.0.2:6379 的槽個(gè)數(shù)

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

      192.168.10.52:6379> CLUSTER INFO cluster_state:fail cluster_slots_assigned:0????# 被分配槽的個(gè)數(shù)為0 cluster_slots_ok:0 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:0 cluster_current_epoch:5 cluster_my_epoch:2 cluster_stats_messages_ping_sent:260418 cluster_stats_messages_pong_sent:260087 cluster_stats_messages_meet_sent:10 cluster_stats_messages_sent:520515 cluster_stats_messages_ping_received:260086 cluster_stats_messages_pong_received:260328 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:520415

      　　
      上面看到集群狀態(tài)是失敗的，原因是槽位沒(méi)有分配，而且需要一次性把16384個(gè)槽位完全分配了，集群才可用。

   2. 分配槽位

      分配槽位：?CLUSTER ADDSLOTS??槽位，一個(gè)槽位只能分配一個(gè)節(jié)點(diǎn)，16384個(gè)槽位必須分配完，不同節(jié)點(diǎn)不能沖突。
      所以通過(guò)腳本進(jìn)行分配 addslots.sh：

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

      #!/bin/bash # node1 192.168.10.52?? 172.17.0.2 n=0 for?((i=n;i<=5461;i++)) do ???/usr/local/bin/redis-cli?-h 192.168.10.52 -p 6379 -a 123456? CLUSTER ADDSLOTS $i done ? ? # node2 192.168.10.52??? 172.17.0.3 n=5462 for?((i=n;i<=10922;i++)) do ???/usr/local/bin/redis-cli?-h 192.168.10.52 -p 6380 -a 123456 CLUSTER ADDSLOTS $i done ? ? # node3 192.168.10.52??? 172.17.0.4 n=10923 for?((i=n;i<=16383;i++)) do ???/usr/local/bin/redis-cli?-h 192.168.10.52 -p 6381 -a 123456 CLUSTER ADDSLOTS $i done

      　　
      其中，?-a?123456? 表示需要輸入的密碼。

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

      192.168.10.52:6379> CLUSTER INFO cluster_state:fail　　　　???????# 集群狀態(tài)為失敗 cluster_slots_assigned:16101????# 沒(méi)有完全分配結(jié)束 cluster_slots_ok:16101 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:2 cluster_stats_messages_ping_sent:266756 cluster_stats_messages_pong_sent:266528 cluster_stats_messages_meet_sent:10 cluster_stats_messages_sent:533294 cluster_stats_messages_ping_received:266527 cluster_stats_messages_pong_received:266666 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:533194<br> 192.168.10.52:6379> CLUSTER INFO cluster_state:ok???????????????????# 集群狀態(tài)為成功 cluster_slots_assigned:16384???????# 已經(jīng)全部分配完成 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:2 cluster_stats_messages_ping_sent:266757 cluster_stats_messages_pong_sent:266531 cluster_stats_messages_meet_sent:10 cluster_stats_messages_sent:533298 cluster_stats_messages_ping_received:266530 cluster_stats_messages_pong_received:266667 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:533198

      　　

      綜上可知，當(dāng)全部槽位分配完成之后，集群還是可行的，如果我們手欠，移除一個(gè)槽位，那么集群就立馬那不行了，自己去試試吧?——CLUSTER DELSLOTS?0?。

5. 如何變成高可用性
   以上我們已經(jīng)搭建了一套完整的可運(yùn)行的redis cluster，但是每個(gè)節(jié)點(diǎn)都是單點(diǎn)，這樣子可能出現(xiàn)，一個(gè)節(jié)點(diǎn)掛掉，整個(gè)集群因?yàn)椴畚环峙洳煌耆罎?#xff0c;因此，我們需要為每個(gè)節(jié)點(diǎn)配置副本備用節(jié)點(diǎn)。
   前面我們已經(jīng)提前創(chuàng)建了6個(gè)備用節(jié)點(diǎn)，搭建集群花了三個(gè)，因此還有剩下三個(gè)直接可以用來(lái)做備用副本。

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

   192.168.10.52:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6???# 總共6個(gè)節(jié)點(diǎn) cluster_size:3??????????# 集群為 3 個(gè)節(jié)點(diǎn) cluster_current_epoch:5 cluster_my_epoch:2 cluster_stats_messages_ping_sent:270127 cluster_stats_messages_pong_sent:269893 cluster_stats_messages_meet_sent:10 cluster_stats_messages_sent:540030 cluster_stats_messages_ping_received:269892 cluster_stats_messages_pong_received:270037 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:539930

   　　
   查看所有節(jié)點(diǎn)的id

   1 2 3 4 5 6 7

   192.168.10.52:6379> CLUSTER NODES 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 172.17.0.3:6379@16379 master - 0 1528704114535 1 connected 5462-10922 f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 172.17.0.4:6379@16379 master - 0 1528704114000 0 connected 10923-16383 ae86224a3bc29c4854719c83979cb7506f37787a 172.17.0.7:6379@16379 master - 0 1528704114023 5 connected 98aebcfe42d8aaa8a3375e4a16707107dc9da683 172.17.0.6:6379@16379 master - 0 1528704115544 4 connected 0bbdc4176884ef0e3bb9b2e7d03d91b0e7e11f44 172.17.0.5:6379@16379 master - 0 1528704114836 3 connected 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 172.17.0.2:6379@16379 myself,master - 0 1528704115000 2 connected 0-5461

   　　
   編寫(xiě)腳本，添加副本節(jié)點(diǎn)

   1 2 3 4 5 6 7 8

   [root@etcd2 tmp]# vi addSlaveNodes.sh #!/bin/bash ? /usr/local/bin/redis-cli?-h 192.168.10.52 -p 6382 -a 123456 CLUSTER REPLICATE 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 ? /usr/local/bin/redis-cli?-h 192.168.10.52 -p 6383 -a 123456 CLUSTER REPLICATE 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c ? /usr/local/bin/redis-cli?-h 192.168.10.52 -p 6384 -a 123456 CLUSTER REPLICATE f45f9109f2297a83b1ac36f9e1db5e70bbc174ab

   　　
   注意：1、作為備用的節(jié)點(diǎn)，必須是未分配槽位的，否者會(huì)操作失敗?(error) ERR To?set?a master the node must be empty and without assigned slots?。
   ? ? ? ? ? ?2、需要從需要添加的節(jié)點(diǎn)上面執(zhí)行操作，CLUSTER REPLICATE [node_id]? ，使當(dāng)前節(jié)點(diǎn)成為 node_id 的副本節(jié)點(diǎn)。
   ? ? ? ? ? ?3、添加從節(jié)點(diǎn)（集群復(fù)制）： 復(fù)制的原理和單機(jī)的Redis復(fù)制原理一樣，區(qū)別是：集群下的從節(jié)點(diǎn)也需要運(yùn)行在cluster模式下，要先添加到集群里面，再做復(fù)制。
   查看所有節(jié)點(diǎn)信息：

   1 2 3 4 5 6 7

   192.168.10.52:6379> CLUSTER NODES 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 172.17.0.3:6379@16379 master - 0 1528705604149 1 connected 5462-10922 f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 172.17.0.4:6379@16379 master - 0 1528705603545 0 connected 10923-16383 ae86224a3bc29c4854719c83979cb7506f37787a 172.17.0.7:6379@16379 slave f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 0 1528705603144 5 connected 98aebcfe42d8aaa8a3375e4a16707107dc9da683 172.17.0.6:6379@16379 slave 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 0 1528705603000 4 connected 0bbdc4176884ef0e3bb9b2e7d03d91b0e7e11f44 172.17.0.5:6379@16379 slave 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 0 1528705603000 3 connected 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 172.17.0.2:6379@16379 myself,master - 0 1528705602000 2 connected 0-5461

   　　
   可以看到我們現(xiàn)在實(shí)現(xiàn)了三主三從的一個(gè)高可用集群。

6. 高可用測(cè)試——故障轉(zhuǎn)移
   查看當(dāng)前運(yùn)行狀態(tài)：

   1 2 3 4 5 6 7

   192.168.10.52:6379> CLUSTER NODES 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 172.17.0.3:6379@16379 master - 0 1528705604149 1 connected 5462-10922 f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 172.17.0.4:6379@16379 master - 0 1528705603545 0 connected 10923-16383 ae86224a3bc29c4854719c83979cb7506f37787a 172.17.0.7:6379@16379 slave f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 0 1528705603144 5 connected 98aebcfe42d8aaa8a3375e4a16707107dc9da683 172.17.0.6:6379@16379 slave 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 0 1528705603000 4 connected 0bbdc4176884ef0e3bb9b2e7d03d91b0e7e11f44 172.17.0.5:6379@16379 slave 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 0 1528705603000 3 connected 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 172.17.0.2:6379@16379 myself,master - 0 1528705602000 2 connected 0-5461

   　　以上，運(yùn)行正常
   
   嘗試關(guān)閉一個(gè)master，選擇端口為6380的容器，停掉之后：

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

   192.168.10.52:6379> CLUSTER NODES 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 172.17.0.3:6379@16379 master,fail - 1528706408935 1528706408000 1 connected 5462-10922 f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 172.17.0.4:6379@16379 master - 0 1528706463000 0 connected 10923-16383 ae86224a3bc29c4854719c83979cb7506f37787a 172.17.0.7:6379@16379 slave f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 0 1528706462980 5 connected 98aebcfe42d8aaa8a3375e4a16707107dc9da683 172.17.0.6:6379@16379 slave 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 0 1528706463000 4 connected 0bbdc4176884ef0e3bb9b2e7d03d91b0e7e11f44 172.17.0.5:6379@16379 slave 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 0 1528706463985 3 connected 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 172.17.0.2:6379@16379 myself,master - 0 1528706462000 2 connected 0-5461 192.168.10.52:6379> 192.168.10.52:6379> CLUSTER INFO cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:10923 cluster_slots_pfail:0 cluster_slots_fail:5461 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:2 cluster_stats_messages_ping_sent:275112 cluster_stats_messages_pong_sent:274819 cluster_stats_messages_meet_sent:10 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:549946 cluster_stats_messages_ping_received:274818 cluster_stats_messages_pong_received:275004 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:549824

   　　以上，發(fā)現(xiàn)整個(gè)集群都失敗了，從節(jié)點(diǎn)沒(méi)有自動(dòng)升級(jí)為主節(jié)點(diǎn)，怎么回事？？
   重啟停掉的容器，經(jīng)排查日志信息?[root@df6ebce6f12a /]# tail -f /var/log/redis/redis-server.log??：

   1 2 3 4 5 6 7 8 9 10

   1:S 11 Jun 09:57:46.712?# Cluster state changed: ok 1:S 11 Jun 09:57:46.718 * (Non critical) Master does not understand REPLCONF listening-port: -NOAUTH Authentication required. 1:S 11 Jun 09:57:46.718 * (Non critical) Master does not understand REPLCONF capa: -NOAUTH Authentication required. 1:S 11 Jun 09:57:46.719 * Partial resynchronization not possible (no cached master) 1:S 11 Jun 09:57:46.719?# Unexpected reply to PSYNC from master: -NOAUTH Authentication required. 1:S 11 Jun 09:57:46.719 * Retrying with SYNC... 1:S 11 Jun 09:57:46.719?# MASTER aborted replication with an error: NOAUTH Authentication required. 1:S 11 Jun 09:57:46.782 * Connecting to MASTER 172.17.0.6:6379 1:S 11 Jun 09:57:46.782 * MASTER <-> SLAVE?sync?started 1:S 11 Jun 09:57:46.782 * Non blocking connect?for?SYNC fired the event.

   
   可以看到，主從之間訪問(wèn)需要auth，之前忘記了配置?redis.conf??中的?# masterauth <master-password>?，所以導(dǎo)致主從之間無(wú)法通訊。修改配置之后，自動(dòng)故障轉(zhuǎn)移正常。

   
   有時(shí)候需要實(shí)施人工故障轉(zhuǎn)移：
   
   登錄6380端口的從節(jié)點(diǎn)：6383，執(zhí)行?CLUSTER FAILOVER?命令：

   1 2	192.168.10.52:6383> CLUSTER? FAILOVER (error) ERR Master is down or failed, please use CLUSTER FAILOVER FORCE

   　　
   發(fā)現(xiàn)因?yàn)閙aster已經(jīng)down了，所以我們需要執(zhí)行強(qiáng)制轉(zhuǎn)移
   ?

   1 2	192.168.10.52:6383> CLUSTER FAILOVER FORCE OK

   　　
   查看當(dāng)前 cluster node 情況：
   ?

   1 2 3 4 5 6 7

   192.168.10.52:6383>? CLUSTER NODES 0bbdc4176884ef0e3bb9b2e7d03d91b0e7e11f44 172.17.0.5:6379@16379 slave 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 0 1528707535332 3 connected ae86224a3bc29c4854719c83979cb7506f37787a 172.17.0.7:6379@16379 slave f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 0 1528707534829 5 connected f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 172.17.0.4:6379@16379 master - 0 1528707534527 0 connected 10923-16383 98aebcfe42d8aaa8a3375e4a16707107dc9da683 172.17.0.6:6379@16379 myself,master - 0 1528707535000 6 connected 5462-10922 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 172.17.0.2:6379@16379 master - 0 1528707535834 2 connected 0-5461 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 172.17.0.3:6379@16379 master,fail - 1528707472833 1528707472000 1 connected

   　　
   從節(jié)點(diǎn)已經(jīng)升級(jí)為master節(jié)點(diǎn)。這時(shí)候，我們嘗試重啟了，6380節(jié)點(diǎn)的redis（其實(shí)是重新啟動(dòng)停掉的容器）：
   ?

   1 2 3 4 5 6 7

   192.168.10.52:6383>? CLUSTER NODES 0bbdc4176884ef0e3bb9b2e7d03d91b0e7e11f44 172.17.0.5:6379@16379 slave 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 0 1528707556044 3 connected ae86224a3bc29c4854719c83979cb7506f37787a 172.17.0.7:6379@16379 slave f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 0 1528707555000 5 connected f45f9109f2297a83b1ac36f9e1db5e70bbc174ab 172.17.0.4:6379@16379 master - 0 1528707556000 0 connected 10923-16383 98aebcfe42d8aaa8a3375e4a16707107dc9da683 172.17.0.6:6379@16379 myself,master - 0 1528707556000 6 connected 5462-10922 760e4d0039c5ac13d04aa4791c9e6dc28544d7c7 172.17.0.2:6379@16379 master - 0 1528707556000 2 connected 0-5461 54cb5c2eb8e5f5aed2d2f7843f75a9284ef6785c 172.17.0.3:6379@16379 slave 98aebcfe42d8aaa8a3375e4a16707107dc9da683 0 1528707556547 6 connected

   　　

   我們發(fā)現(xiàn)，6380節(jié)點(diǎn)反而變成了 6383節(jié)點(diǎn)的從節(jié)點(diǎn)。
   
   現(xiàn)在集群應(yīng)該是完整的了，所以，集群狀態(tài)應(yīng)該已經(jīng)恢復(fù)了，我們查看下：

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

   192.168.10.52:6383> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:6 cluster_stats_messages_ping_sent:19419 cluster_stats_messages_pong_sent:19443 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:38869 cluster_stats_messages_ping_received:19433 cluster_stats_messages_pong_received:19187 cluster_stats_messages_meet_received:5 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:38631

   　　

   OK，沒(méi)有問(wèn)題。
   ?
7. 集群訪問(wèn)
   客戶(hù)端在初始化的時(shí)候只需要知道一個(gè)節(jié)點(diǎn)的地址即可，客戶(hù)端會(huì)先嘗試向這個(gè)節(jié)點(diǎn)執(zhí)行命令，比如 ?get?key?，如果key所在的slot剛好在該節(jié)點(diǎn)上，則能夠直接執(zhí)行成功。如果slot不在該節(jié)點(diǎn)，則節(jié)點(diǎn)會(huì)返回MOVED錯(cuò)誤，同時(shí)把該slot對(duì)應(yīng)的節(jié)點(diǎn)告訴客戶(hù)端，客戶(hù)端可以去該節(jié)點(diǎn)執(zhí)行命令

   1 2 3 4 5

   192.168.10.52:6383> get hello (error) MOVED 866 172.17.0.2:6379 ? 192.168.10.52:6379>?set?number 20004 (error) MOVED 7743 172.17.0.3:6379

   　　另外，redis集群版只使用db0，select命令雖然能夠支持select 0。其他的db都會(huì)返回錯(cuò)誤。

   1 2 3 4

   192.168.10.52:6383>?select?0 OK 192.168.10.52:6383>?select?1 (error) ERR SELECT is not allowed?in?cluster mode

   ?

?

1. 近期，有網(wǎng)友詢(xún)問(wèn)docker redis集群連接報(bào)錯(cuò)的問(wèn)題，具體報(bào)錯(cuò)如下：
   
   　

   ?

   初步認(rèn)為是，node節(jié)點(diǎn)沒(méi)有全部添加進(jìn)去，添加之后，依然有上述問(wèn)題。想到是跨主機(jī)訪問(wèn)，應(yīng)該是路由尋址不了導(dǎo)致的。當(dāng)初寫(xiě)上述教程的時(shí)候，docker是以默認(rèn)的網(wǎng)絡(luò)模式bridge模式運(yùn)行的，畢竟當(dāng)初是以學(xué)習(xí)整理文檔為主，主要是單機(jī)訪問(wèn)。但是，實(shí)際應(yīng)用化場(chǎng)景中，多是公網(wǎng)跨主機(jī)訪問(wèn)，問(wèn)題明朗了，想著集群這東西最好還是設(shè)置成共享主機(jī)公網(wǎng)ip比較好，于是解決如下：
   1. 在docker運(yùn)行時(shí)，執(zhí)行網(wǎng)絡(luò)模式為：host。
   2. 端口沖突解決，畢竟host模式下，容器會(huì)占用宿主機(jī)的端口，于是，我們就從配置下手，在宿主機(jī)上生成配置redis-60001.conf,redis-60002.conf,redis-60003.conf...，有多少端口建多少個(gè)文件，最終運(yùn)行一個(gè)容器，掛載一個(gè)配置到容器中用于覆蓋主機(jī)中的配置。
      最終的運(yùn)行方式如下：

      1	docker run -d --name redis-6380 --net host -v?/tmp/redis.conf:/usr/local/redis/redis.conf? hakimdstx/nodes-redis:4.0.1

      
      至此，網(wǎng)絡(luò)問(wèn)題得到解決。
      PS.生產(chǎn)環(huán)境需要注意防火墻問(wèn)題，不然也是會(huì)報(bào)錯(cuò)的。　

總結(jié)

以上是生活随笔為你收集整理的docker 实现redis集群搭建的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。