Deployment Modes of Redis

Redis can be operated in various setups depending on your application's scalability, availability, and fault-tolerance needs. Below is a summary of each mode:

Standalone

The simplest and default mode.

A single Redis instance handles all reads and writes.

Best for development, testing, or low-traffic production environments.

Pros	Cons
Easy to set up and maintain	Single point of failure
High performance, single-node does not need to synchronize data, and data has natural consistency	No high availability or fault tolerance (When you don't need Redis to keep running in case of a crash)

Master-Slave (Replication)

Master-Slave mode consists of N Redis instances, which can be one master with N slaves (if N masters with N slaves, it is not strictly a master-slave mode, and we will discuss it in the cluster mode. N+N Redis instances are required for N masters with N slaves). One function of the master-slave mode is to back up data so that data can be easily recovered when a node is damaged (meaning irreparable hardware damage) because there is a backup. Another function is load balancing. If all clients access one node, it will affect the efficiency of Redis's work. With master-slave, query operations can be completed by querying the slave node. Since master-slave replication means that the data of the master and slave are the same, there is a problem of data redundancy.

Pros	Cons
Expand the read ability of the master node and share the read pressure of the master node	A single machine limits the write ability and storage capacity of the master node
The cornerstone of high availability	Once the master node fails, the slave node needs to be promoted as the new master node, and the application's master node address needs to be modified. It is also necessary to command all slave nodes to replicate the new master node, which requires manual intervention throughout the process

Sentinel

In a master-slave mode, when the master node goes down, the slave node can take over as the master node and continue providing service. However, there is an issue when the IP address of the master node changes. The application service still uses the original master node address to access, requiring manual intervention to modify. Sentinel can precisely solve this problem. Access to data in the Redis cluster is through the Sentinel cluster, which monitors the entire Redis cluster. Once a problem is detected in the Redis cluster, such as the master node going down, the slave node takes over. But when the master node address changes, the application service is unaware and does not need to change the access address because Sentinel interacts with the application service. Sentinel solves the failover problem well, taking high availability to another level. Of course, Sentinel has other functions, such as master node live detection, master-slave operation detection, and master-slave switching. A practical minimum configuration for Redis Sentinel is one master and one slave, monitored by at least three Sentinel nodes to ensure quorum and avoid split-brain.

Pros	Cons
All the advantages of the master-slave mode	Slightly more complex setup
High availability	Requires configuration of Sentinel nodes

Cluster

While master-slave replication provides data redundancy and Sentinel ensures automatic failover, these setups still have limitations. A single Redis node’s storage and access capacity is restricted, and scaling write operations across multiple nodes is not possible. Redis Cluster addresses these challenges by offering high availability, horizontal scalability, distribution, and fault tolerance.

For production-ready deployments, a Redis Cluster requires at least three master nodes, each of which can optionally have one or more replica nodes. This ensures data redundancy and allows the cluster to tolerate node failures without downtime. Cluster mode also enables automatic sharding, where data is divided across nodes, allowing the system to handle larger datasets and higher request throughput than single-node setups.

How does it work?

Data sharing is achieved through data sharding, while providing data replication and fault transfer functions. In the previous two modes, data was all on one node, and the storage capacity of a single node is limited. Cluster mode shards data and stores it on multiple nodes. When a shard reaches its limit, it is divided into various shards. Data is divided into 16384 slots (hash slots) in the key space of the cluster, and data is distributed to different shards based on hash, as follows:

HASH_SLOT = CRC16(key) % 16384

Data reading and writing after sharding: read requests are assigned to slave nodes, and write requests are assigned to the master node. Data is synchronized from the master to the slave node. Separating read and write requests improves concurrency and increases performance.

Horizontal expansion after data sharding: The master node can be expanded, and data migration is automatically completed within Redis. When you add a new master node, data migration is required, but the Redis service does not need to be offline.

For example, there are three master nodes, which means that Redis slots are divided into three segments, assuming the three segments are 0-7000, 7001-12000, and 12001-16383, respectively. Due to business needs, a new master node is added, and the four nodes collectively occupy 16384 slots. The slots need to be re-assigned, and the data must be migrated, but the service does not need to be offline. The Redis-trib management software within Redis performs the Redis cluster's re-sharding. Redis provides all the commands for re-sharding, and Redis-trib performs re-sharding by sending commands to the nodes.

Why does the Redis cluster have 16384 slots?

If the slot is 65536, the message header for sending heartbeat messages is too large, reaching 8 KB. As mentioned above, the largest space in the message header is myslots[CLUSTER_SLOTS/8]. When the slot is 65536, the size of this block is 8KB: 65536÷8÷1024=8KB. Since Redis nodes need to send a certain number of ping messages as heartbeat messages every second, if the slot is 65536, the message header of this ping message is too large and wastes bandwidth.

The number of Redis cluster master nodes cannot exceed 1000. As mentioned above, the more cluster nodes there are, the more data is carried in the message body of the heartbeat package. If there are more than 1000 nodes, it will also cause network congestion. Therefore, having more than 1000 Redis cluster nodes is not recommended. For Redis clusters with less than 1000 nodes, 16384 slots are enough. There is no need to expand to 65536. The smaller the slot, the higher the compression ratio with fewer nodes. In the Redis master node's configuration information, the hash slot it is responsible for is saved as a bitmap. During transmission, the bitmap is compressed. However, if the filling rate of the bitmap slots/N is high (N represents the number of nodes), the compression rate of the bitmap is low. If there are few nodes and many hash slots, the compression rate of the bitmap is low.

Example docker

name: redis-cluster
services:
  redis-cluster-0:
    container_name: redis-cluster-0
    image: bitnami/redis-cluster:latest
    ports:
      - 6379:6379
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6379'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6379'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-1:
    container_name: redis-cluster-1
    image: bitnami/redis-cluster:latest
    ports:
      - 6380:6380
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6380'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6380'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-2:
    container_name: redis-cluster-2
    image: bitnami/redis-cluster:latest
    ports:
      - 6381:6381
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6381'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6381'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-3:
    container_name: redis-cluster-3
    image: bitnami/redis-cluster:latest
    ports:
      - 6382:6382
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6382'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6382'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-4:
    container_name: redis-cluster-4
    image: bitnami/redis-cluster:latest
    ports:
      - 6383:6383
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6383'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6383'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-5:
    container_name: redis-cluster-5
    image: bitnami/redis-cluster:latest
    ports:
      - 6384:6384
    depends_on:
      - redis-cluster-0
      - redis-cluster-1
      - redis-cluster-2
      - redis-cluster-3
      - redis-cluster-4
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6384'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6384'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
      - 'REDIS_CLUSTER_REPLICAS=1'
      - 'REDIS_CLUSTER_CREATOR=yes'
    networks:
      - redis-network

  redis-insight:
    container_name: redis-insight
    image: redis/redisinsight:latest
    ports:
      - 5540:5540
    networks:
      - redis-network

networks:
  redis-network:
    driver: bridge

Helpful Definitions

Term	Meaning
High Availability	The system keeps working even if some parts stop
Fault Tolerance	The ability of a system to continue working even if some parts fail. It prevents total system failure
Failover	Switching to a backup system if the main one fails
Replication	Making copies of data from one server to another
Throughput	The amount of work or data a system can process in a given amount of time.
Latency	The time it takes for a request to travel from sender to receiver and get a response.
Scalability	The system’s ability to handle increased load by adding resources.