How to deploy redis cluster in k8s
redis cluster construction
1.1 Use redis-cli to create a cluster
# 查看redis的pod对应的ip
kubectl get pod -n jxbp -o wide
>NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
redis-0 1/1 Running 0 18h 10.168.235.196 k8s-master <none> <none>
redis-1 1/1 Running 0 18h 10.168.235.225 k8s-master <none> <none>
redis-2 1/1 Running 0 18h 10.168.235.239 k8s-master <none> <none>
redis-3 1/1 Running 0 18h 10.168.235.198 k8s-master <none> <none>
redis-4 1/1 Running 0 18h 10.168.235.222 k8s-master <none> <none>
redis-5 1/1 Running 0 18h 10.168.235.238 k8s-master <none> <none>
# 进入到redis-0容器
kubectl exec -it redis-0 /bin/bash -n jxbp
# 创建master节点（redis-0、redis-2、redis-4）
redis-cli --cluster create 10.168.235.196:6379 10.168.235.239:6379 10.168.235.222:6379 -a jxbd
> Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
M: bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 10.168.235.196:6379
slots:[0-5460] (5461 slots) master
M: 4367e4a45e557406a3112e7b79f82a44d4ce485e 10.168.235.239:6379
slots:[5461-10922] (5462 slots) master
M: a2cec159bbe2efa11a8f60287b90927bcb214729 10.168.235.222:6379
slots:[10923-16383] (5461 slots) master
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 10.168.235.196:6379)
M: bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 10.168.235.196:6379
slots:[0-5460] (5461 slots) master
M: a2cec159bbe2efa11a8f60287b90927bcb214729 10.168.235.222:6379
slots:[10923-16383] (5461 slots) master
M: 4367e4a45e557406a3112e7b79f82a44d4ce485e 10.168.235.239:6379
slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.Note that the master node above will generate the corresponding node id: bcae187137a9b30d7dab8fe0d8ed4a46c6e39638, a2cec159bbe2efa11a8f60287b90927bcb21 4729 , 4367e4a45e557406a3112e7b79f82a44d4ce485e, used to create slave nodes.
# 为每个master节点添加slave节点 # 10.168.235.196:6379的位置可以是任意一个master节点,一般我们用第一个master节点即redis-0的ip地址 # --cluster-master-id参数指定该salve节点对应的master节点的id # -a参数指定redis的密码 # redis-0的master节点，添加redis-1为slave节点 redis-cli --cluster add-node 10.168.235.225:6379 10.168.235.196:6379 --cluster-slave --cluster-master-id bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 -a jxbd # redis-2的master节点，添加redis-3为slave节点 redis-cli --cluster add-node 10.168.235.198:6379 10.168.235.239:6379 --cluster-slave --cluster-master-id a2cec159bbe2efa11a8f60287b90927bcb214729 -a jxbd # redis-4的master节点，添加redis-5为slave节点 redis-cli --cluster add-node 10.168.233.238:6379 10.168.235.222:6379 --cluster-slave --cluster-master-id 4367e4a45e557406a3112e7b79f82a44d4ce485e -a jxbd
The following information is displayed, which means the addition is successful:
[OK] All nodes agree about slots configuration.
[OK] All 16384 slots covered.
[OK] New node added correctly.
Pitfall:
At first, I wanted to use the headless domain name to create a redis cluster, so that there would be no need to update the IP after the node restarts, but redis does not It supports the use of domain names, so it can only go around in a circle and return to the fixed IP method, which is very inconsistent with the container environment.
1.2redis cluster status verification (optional)
cluster info
# 进入到redis客户端，集群需要带上-c，有密码需要带上-a redis-cli -c -a jxbd # 查看redis集群信息 127.0.0.1:6379> cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:7996 cluster_stats_messages_pong_sent:7713 cluster_stats_messages_sent:15709 cluster_stats_messages_ping_received:7710 cluster_stats_messages_pong_received:7996 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:15709
Note:
Now you can access the Redis service from any Pod in the cluster. Earlier we created a headless Service. The kubernetes cluster will assign a DNS record to the service in the format: $(pod.name).$(headless server.name).${namespace}.svc.cluster.local, each time the service name is accessed, it will be directly Enter the redis node. svc.cluster.localcan be omitted. For example:
redis-cli -c -a jxbd -h redis-0.redis-hs.jxbp -p 6379
cluster nodes
# 查看redis集群状态 127.0.0.1:6379> cluster nodes 70220b45e978d0cb3df19b07e55d883b49f4127d 10.168.235.238:6379@16379 slave 4367e4a45e557406a3112e7b79f82a44d4ce485e 0 1670306292673 2 connected 122b89a51a9bf005e3d47b6d721c65621d2e9a75 10.168.235.225:6379@16379 slave bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 0 1670306290558 1 connected c2afcb9e83038a47d04bf328ead8033788548234 10.168.235.198:6379@16379 slave a2cec159bbe2efa11a8f60287b90927bcb214729 0 1670306291162 3 connected 4367e4a45e557406a3112e7b79f82a44d4ce485e 10.168.235.239:6379@16379 master - 0 1670306291561 2 connected 5461-10922 bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 10.168.235.196:6379@16379 myself,master - 0 1670306291000 1 connected 0-5460 a2cec159bbe2efa11a8f60287b90927bcb214729 10.168.235.222:6379@16379 master - 0 1670306292166 3 connected 10923-16383
You can see 3 master and 3 slave nodes, all in the connected status.
get, set verification
# 会找到对应的槽进行set操作，去到10.168.235.222节点 set name1 llsydn -> Redirected to slot [12933] located at 10.168.235.222:6379 OK # set name1成功 10.168.235.222:6379> set name1 llsydn OK # get name1成功 10.168.235.222:6379> get name1 "llsydn"
The master node performs the set operation and the slave node replicates. Master-slave replication
1.3 Restart the pod and verify the cluster (optional)
# redis-1未重启之前 10.168.235.239:6379> cluster nodes 4367e4a45e557406a3112e7b79f82a44d4ce485e 10.168.235.239:6379@16379 myself,master - 0 1670307319000 2 connected 5461-10922 bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 10.168.235.196:6379@16379 master - 0 1670307319575 1 connected 0-5460 70220b45e978d0cb3df19b07e55d883b49f4127d 10.168.235.238:6379@16379 slave 4367e4a45e557406a3112e7b79f82a44d4ce485e 0 1670307318000 2 connected 122b89a51a9bf005e3d47b6d721c65621d2e9a75 10.168.235.225:6379@16379 slave bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 0 1670307319781 1 connected c2afcb9e83038a47d04bf328ead8033788548234 10.168.235.198:6379@16379 slave a2cec159bbe2efa11a8f60287b90927bcb214729 0 1670307319071 3 connected a2cec159bbe2efa11a8f60287b90927bcb214729 10.168.235.222:6379@16379 master - 0 1670307318000 3 connected 10923-16383 # 重启redis-1 kubectl delete pod redis-1 -n jxbp pod "redis-1" deleted # redis-1重启之后 10.168.235.239:6379> cluster nodes 4367e4a45e557406a3112e7b79f82a44d4ce485e 10.168.235.239:6379@16379 myself,master - 0 1670307349000 2 connected 5461-10922 bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 10.168.235.196:6379@16379 master - 0 1670307349988 1 connected 0-5460 70220b45e978d0cb3df19b07e55d883b49f4127d 10.168.235.238:6379@16379 slave 4367e4a45e557406a3112e7b79f82a44d4ce485e 0 1670307349000 2 connected 122b89a51a9bf005e3d47b6d721c65621d2e9a75 10.168.235.232:6379@16379 slave bcae187137a9b30d7dab8fe0d8ed4a46c6e39638 0 1670307350089 1 connected c2afcb9e83038a47d04bf328ead8033788548234 10.168.235.198:6379@16379 slave a2cec159bbe2efa11a8f60287b90927bcb214729 0 1670307350000 3 connected a2cec159bbe2efa11a8f60287b90927bcb214729 10.168.235.222:6379@16379 master - 0 1670307348000 3 connected 10923-16383
You can see the redis-1 node after the restart. Although the IP has changed, the redis cluster can still be recognized. To the new IP, the cluster is still normal.
10.168.235.225 ---> 10.168.235.232
1.4 Create Service Service
Earlier we created a Headless Service for implementing StatefulSet, but the Service does not have a Cluster IP and therefore cannot be used for external access. Therefore, we need to create a Service specifically to provide access and load balancing to the Redis cluster.
You can use ClusterIP, NodePort here. Here, I am using NodePort.
vi redis-ss.yaml
---
apiVersion: v1
kind: Service
metadata:
labels:
k8s.kuboard.cn/layer: db
k8s.kuboard.cn/name: redis
name: redis-ss
namespace: jxbp
spec:
ports:
- name: imdgss
port: 6379
protocol: TCP
targetPort: 6379
nodePort: 6379
selector:
k8s.kuboard.cn/layer: db
k8s.kuboard.cn/name: redis
type: NodePortCreate a service named: redis-ss.
Expose port 6379 in the K8S cluster, and load balance the pods with labels name being k8s.kuboard.cn/name: redis.
Then in the K8S cluster, you can access the redis cluster through redis-ss:6379.
kubectl get service -n jxbp >NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE redis-hs ClusterIP None <none> 6379/TCP 76m redis-ss NodePort 10.96.54.201 <none> 6379:6379/TCP 2s
1.5 Springboot project configuration
spring.redis.cluster.nodes=redis-ss:6379
1.6 Analysis of related questions
At this point, you may be wondering why the stable flag is not used and the Redis Pod can also fail normally. What about transfer? This involves the mechanism of Redis itself. Because each node in the Redis cluster has its own NodeId (saved in the automatically generated nodes.conf), and the NodeId will not change with the IP. This is actually a fixed network symbol. In other words, even if a Redis Pod is restarted due to a reboot, the Pod will still use the saved NodeId to maintain its identity. You can view the redis-0 node configuration file nodes.conf
vi /opt/nfs/pv1/nodes.conf > f6d4993467a4ab1f3fa806f1122edd39f6466394 10.168.235.228:6379@16379 slave ebed24c8fca9ebc16ceaaee0c2bc2e3e09f7b2c0 0 1670316449064 2 connected ebed24c8fca9ebc16ceaaee0c2bc2e3e09f7b2c0 10.168.235.240:6379@16379 myself,master - 0 1670316450000 2 connected 5461-10922 955e1236652c2fcb11f47c20a43149dcd1f1f92b 10.168.235.255:6379@16379 master - 0 1670316449565 1 connected 0-5460 574c40485bb8f6cfaf8618d482efb06f3e323f88 10.168.235.224:6379@16379 slave 955e1236652c2fcb11f47c20a43149dcd1f1f92b 0 1670316449000 1 connected 91bd3dc859ce51f1ed0e7cbd07b13786297bd05b 10.168.235.237:6379@16379 slave fe0b74c5e461aa22d4d782f891b78ddc4306eed4 0 1670316450672 3 connected fe0b74c5e461aa22d4d782f891b78ddc4306eed4 10.168.235.253:6379@16379 master - 0 1670316450068 3 connected 10923-16383 vars currentEpoch 3 lastVoteEpoch 0
through NFS. As above, the first column is NodeId, which is stable; the second column is IP and port information, which may change.
Here, we introduce two usage scenarios of NodeId:
When a Slave Pod is disconnected and reconnected, the IP changes, but the Master finds that its NodeId is still the same, so it thinks that the Slave is still the same as before. Slave.
If a Master Pod is disconnected, other Slaves in the cluster will elect a new Master. If the old Master comes online and the cluster finds that its NodeId is still the same, it will become the slave node of the new Master.
The above is the detailed content of How to deploy redis cluster in k8s. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1382
52
How to build the redis cluster mode
Apr 10, 2025 pm 10:15 PM
Redis cluster mode deploys Redis instances to multiple servers through sharding, improving scalability and availability. The construction steps are as follows: Create odd Redis instances with different ports; Create 3 sentinel instances, monitor Redis instances and failover; configure sentinel configuration files, add monitoring Redis instance information and failover settings; configure Redis instance configuration files, enable cluster mode and specify the cluster information file path; create nodes.conf file, containing information of each Redis instance; start the cluster, execute the create command to create a cluster and specify the number of replicas; log in to the cluster to execute the CLUSTER INFO command to verify the cluster status; make
How to clear redis data
Apr 10, 2025 pm 10:06 PM
How to clear Redis data: Use the FLUSHALL command to clear all key values. Use the FLUSHDB command to clear the key value of the currently selected database. Use SELECT to switch databases, and then use FLUSHDB to clear multiple databases. Use the DEL command to delete a specific key. Use the redis-cli tool to clear the data.
How to use the redis command
Apr 10, 2025 pm 08:45 PM
Using the Redis directive requires the following steps: Open the Redis client. Enter the command (verb key value). Provides the required parameters (varies from instruction to instruction). Press Enter to execute the command. Redis returns a response indicating the result of the operation (usually OK or -ERR).
How to use redis lock
Apr 10, 2025 pm 08:39 PM
Using Redis to lock operations requires obtaining the lock through the SETNX command, and then using the EXPIRE command to set the expiration time. The specific steps are: (1) Use the SETNX command to try to set a key-value pair; (2) Use the EXPIRE command to set the expiration time for the lock; (3) Use the DEL command to delete the lock when the lock is no longer needed.
How to read redis queue
Apr 10, 2025 pm 10:12 PM
To read a queue from Redis, you need to get the queue name, read the elements using the LPOP command, and process the empty queue. The specific steps are as follows: Get the queue name: name it with the prefix of "queue:" such as "queue:my-queue". Use the LPOP command: Eject the element from the head of the queue and return its value, such as LPOP queue:my-queue. Processing empty queues: If the queue is empty, LPOP returns nil, and you can check whether the queue exists before reading the element.
How to implement the underlying redis
Apr 10, 2025 pm 07:21 PM
Redis uses hash tables to store data and supports data structures such as strings, lists, hash tables, collections and ordered collections. Redis persists data through snapshots (RDB) and append write-only (AOF) mechanisms. Redis uses master-slave replication to improve data availability. Redis uses a single-threaded event loop to handle connections and commands to ensure data atomicity and consistency. Redis sets the expiration time for the key and uses the lazy delete mechanism to delete the expiration key.
How to read the source code of redis
Apr 10, 2025 pm 08:27 PM
The best way to understand Redis source code is to go step by step: get familiar with the basics of Redis. Select a specific module or function as the starting point. Start with the entry point of the module or function and view the code line by line. View the code through the function call chain. Be familiar with the underlying data structures used by Redis. Identify the algorithm used by Redis.
How to make message middleware for redis
Apr 10, 2025 pm 07:51 PM
Redis, as a message middleware, supports production-consumption models, can persist messages and ensure reliable delivery. Using Redis as the message middleware enables low latency, reliable and scalable messaging.
