Redis集群节点的选举:

    当master挂掉之后，就会在该集群中的slave中选取一个来代替mater角色，

    从而保证redis集群slot的完整性。

    如果其中一个mster和它的slave都挂掉后，会导致slot不完整，整个集群都会挂掉。

集群节点信息：

    192.168.2.200:6379> cluster nodes

    3ff3a74f9dc41f8bc635ab845ad76bf77ffb0f69 192.168.2.201:6379 master - 0 1527145806504 5 connected 10923-16383

    2faf68564a70372cfc06c1afff197019cc6a39f3 192.168.2.201:6380 slave 3ff3a74f9dc41f8bc635ab845ad76bf77ffb0f69 0 1527145805495 5 connected

    227f51028bbe827f27b4e40ed7a08fcc7d8df969 192.168.2.200:6380 slave 098e7eb756b6047fde988ab3c0b7189e1724ecf5 0 1527145804485 4 connected

    5844b4272c39456b0fdf73e384ff8c479547de47 192.168.2.200:6379 myself,master - 0 0 3 connected 5461-10922

    7119dec91b086ca8fe69f7878fa42b1accd75f0f 192.168.2.100:6380 slave 5844b4272c39456b0fdf73e384ff8c479547de47 0 1527145802468 3 connected

    098e7eb756b6047fde988ab3c0b7189e1724ecf5 192.168.2.100:6379 master - 0 1527145803476 1 connected 0-5460

新建3个key：

    192.168.2.200:6379> set name zhangsan

    OK

    192.168.2.200:6379> set age 26

    -> Redirected to slot [741] located at 192.168.2.100:6379

    OK

    192.168.2.100:6379> set home beijing

    -> Redirected to slot [10814] located at 192.168.2.200:6379

    OK

模拟192.168.2.100:6379挂掉：

    # ps -ef |grep redis

        root      19023      1  0 15:05 ?        00:00:01 redis-server 192.168.2.100:6379 [cluster]

        root      19030      1  0 15:05 ?        00:00:01 redis-server 192.168.2.100:6380 [cluster]

        root      19127   2912  0 15:13 pts/0    00:00:00 grep --color=auto redis

    # kill 19023

查看集群节点状态：（可以看到集群中的一个slave的角色变成了master）

    # redis-trib.rb check 192.168.2.200:6379

    >>> Performing Cluster Check (using node 192.168.2.200:6379)

    M: 5844b4272c39456b0fdf73e384ff8c479547de47 192.168.2.200:6379

    slots:5461-10922 (5462 slots) master

    1 additional replica(s)

    M: 3ff3a74f9dc41f8bc635ab845ad76bf77ffb0f69 192.168.2.201:6379

    slots:10923-16383 (5461 slots) master

    1 additional replica(s)

    S: 2faf68564a70372cfc06c1afff197019cc6a39f3 192.168.2.201:6380

    slots: (0 slots) slave

    replicates 3ff3a74f9dc41f8bc635ab845ad76bf77ffb0f69

    M: 227f51028bbe827f27b4e40ed7a08fcc7d8df969 192.168.2.200:6380

    slots:0-5460 (5461 slots) master

    0 additional replica(s)

    S: 7119dec91b086ca8fe69f7878fa42b1accd75f0f 192.168.2.100:6380

    slots: (0 slots) slave

    replicates 5844b4272c39456b0fdf73e384ff8c479547de47

    [OK] All nodes agree about slots configuration.

    >>> Check for open slots...

    >>> Check slots coverage...

    [OK] All 16384 slots covered.

查看数据是否存在：

    # redis-cli -h 192.168.2.200 -p 6380 -c

        192.168.2.200:6380> keys *

        1) "age"

模拟恢复故障节点：

    # redis-server redis.conf

查看集群节点状态：（故障节点恢复后，是变成master还是slave？）

    # redis-trib.rb check 192.168.2.200:6379

    >>> Performing Cluster Check (using node 192.168.2.200:6379)

    M: 5844b4272c39456b0fdf73e384ff8c479547de47 192.168.2.200:6379

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

    M: 3ff3a74f9dc41f8bc635ab845ad76bf77ffb0f69 192.168.2.201:6379

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

    S: 2faf68564a70372cfc06c1afff197019cc6a39f3 192.168.2.201:6380

   slots: (0 slots) slave

   replicates 3ff3a74f9dc41f8bc635ab845ad76bf77ffb0f69

    M: 227f51028bbe827f27b4e40ed7a08fcc7d8df969 192.168.2.200:6380

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

    S: 7119dec91b086ca8fe69f7878fa42b1accd75f0f 192.168.2.100:6380

   slots: (0 slots) slave

   replicates 5844b4272c39456b0fdf73e384ff8c479547de47

    S: 098e7eb756b6047fde988ab3c0b7189e1724ecf5 192.168.2.100:6379

   slots: (0 slots) slave

   replicates 227f51028bbe827f27b4e40ed7a08fcc7d8df969

    [OK] All nodes agree about slots configuration.

    >>> Check for open slots...

    >>> Check slots coverage...

    [OK] All 16384 slots covered.

总结：master：192.168.2.100:6379 宕机后，

      slave：192.168.2.200:6380 代替了其位置，变成了master角色。

      当192.168.2.100:6379故障恢复后，并没有恢复到master的角色，

      而是充当了slave角色。