Redis Clustering With Redis Sentinel On Docker

The most of us need caching systems for the projects we work onit. Redis is the most popluar of them and it is great.

Of course, just installing and running “redis” seem to be enough, but sometimes cause scalability and accessibility problems. So there is a tool named “redis-sentinel” to manage “redis” services.

What are the capabilities of “redis-sentinel”?

Monitoring: It can check master or slave services whether is working correctly and healthy or not.

Notification: It can notify when a service goes down.

Automatic Failover: It starts a failover scenario and assigns one of slaves as a master when master service goes down.

Configuration Provider: It can configure and share master service information when a new service is added to cluster.

Sentinel is not a “redis server”. So “redis-server” commands don’t work on “redis-sentinel”.

So if you will use “redis-sentinel”, first you should ask master service to “redis-sentinel”, then run redis command on “redis-master”.

That’s enough chatting, time to implementation :)

First of all you should install “docker” and “docker-compose” to your system. Then you should checkout source repository of docker compose configuration from here:

mustafaileri/redis-cluster-with-sentinel

You will see 1 master, 1 slave “redis” and 1 “sentinel” when you examine into services that included docker-compose.yml

Now we can start docker containers with this command.

docker-compose up --build

Check “docker” containers when build process is completed.

docker-compose ps

You should see 3 containers and these should be up when you run command on above.

First, create new slaves. We can use “scale” command.

docker-compose scale slave=4

Then, create new sentinels via “scale” command.

docker-compose scale sentinel=3

Now, we have 4 slaves and 3 sentinels. That’s great.

At this point, if master service goes down:

1. Each “sentinel” service verifies that master is down. When specified number of “sentinel” service(quorum) verifies that master is down, goes next step.
2. Sentinels will wait that the master service goes up again on the specified time. “Automatic Failover” starts if master doesn’t go to up.
   Sentinel services select from one of slaves as master by election.
3. Finally selected master service configuration changes are distributed to all “redis” services.

Let’s try this scenario.

First ask to “sentinel” for master “redis” service.

docker-compose exec sentinel redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster

Sentinel returns; master service ip and master service port.

Wait! What is the “mymaster”?
This is registered name of master service on sentinel. For more info:
https://github.com/mustafaileri/redis-cluster-with-sentinel/blob/master/sentinel/sentinel.conf#L3

Now stop the master service:

docker-compose pause master

Failover scenario will start when “DOWN-AFTER” and “FAIL AFTER” times are completed and new master will be assigned. You can get logs created while this process with this command.

docker-compose logs sentinel

Now, we see “mymaster” information changed when we ask again “mymaster” information to “sentinel”. Because master service goes down and new master service assigned from slaves.

If you are a developer and you are not interested these details you can use these libraries for python and php.

nrk/predis

redis 2.10.5 : Python Package Index