Question posted in
The official documentation can be found here.

Spring Boot Caching with Redis Sentinel always connects to master node

I have a Spring Boot (2.3.1.RELEASE) app that has caching with Redis Sentinel.
This is my configuration for the Sentinel connection:

@Bean
public LettuceConnectionFactory redisConnectionFactory() {
   RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
            .master(redisProperties.getSentinel().getMaster());
    redisProperties.getSentinel().getNodes().forEach(s -> sentinelConfig.sentinel(s, redisProperties.getPort()));
    sentinelConfig.setPassword(RedisPassword.of(redisProperties.getPassword()));
    return new LettuceConnectionFactory(sentinelConfig);
}

And this is my caching manager configuration:

@Bean
public RedisCacheManager cacheManager() {
    Map<String, RedisCacheConfiguration> cacheConfigs = new HashMap<>();
    cacheConfigs.put("cache1", RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofMinutes(ttlMinutes)));
    cacheConfigs.put("cache2", RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofMinutes(ttlMinutes)));

    return RedisCacheManager.builder(redisConnectionFactory())
            .cacheDefaults(RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofMinutes(ttlMinutes)))
            .withInitialCacheConfigurations(cacheConfigs)
            .transactionAware()
            .build();
}

Everything works fine from caching perspective.

However, If I turn on debug logs inside the io.lettuce.core.protocol.CommandHandler, I see that it is always connecting to the same node (master). Which I can confirm by looking at the logs on the node.

Everywhere I look online, this seems like the correct configuration.

This brings me to my question:

  • Is it possible to configure the Spring caching abstraction to use the master node only for writes and the slave nodes for reads?

Is this expectation even valid? Or this is the way Sentinel is supposed to be used (all requests go to master)?

Tags:

2

Answers


  1. 0

    Yes, it can be done.

    From Spring Data Redis Docs – 10.4.4. Write to Master, Read from Replica :

    It is said that Spring Data Redis provides a Redis Master/Replica setup which not
    only allows data to be safely stored at more nodes but also allows
    reading data from replicas while pushing writes to the master by using
    Lettuce.

    For this, you have to update redisConnectionFactory() method in the configuration class :

     @Bean
 public LettuceConnectionFactory redisConnectionFactory() {
   LettuceClientConfiguration clientConfig = LettuceClientConfiguration.builder()
            .readFrom(ReadFrom.REPLICA_PREFERRED)
            .build();
   RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
            .master(redisProperties.getSentinel().getMaster());
   redisProperties.getSentinel().getNodes().forEach(s -> sentinelConfig.sentinel(s, redisProperties.getPort()));
   sentinelConfig.setPassword(RedisPassword.of(redisProperties.getPassword()));
   return new LettuceConnectionFactory(sentinelConfig, clientConfig);
}
    Login or Signup to reply.
  2. 0

    if you want to write master and read only from all slaves(replicas), you can use below configuration.

    These configurations include connection pooling,write to master and read from all slave nodes with round-robin load balancing.

        @Bean
    public RedisConnectionFactory lettuceConnectionFactory(RedisProperties redisProperties, LettucePoolingClientConfiguration lettucePoolingClientConfiguration) {
        final RedisSentinelConfiguration redisSentinelConfiguration = new RedisSentinelConfiguration().master(redisProperties.getMaster());
        redisSentinelConfiguration.setDatabase(redisProperties.getDbIndex());
        addSentinels(redisProperties, redisSentinelConfiguration);
        return new LettuceConnectionFactory(redisSentinelConfiguration, lettucePoolingClientConfiguration);
    }

    private void addSentinels(RedisProperties redisProperties, RedisSentinelConfiguration redisSentinelConfiguration) {
        redisProperties.getNodes()
                .forEach(node -> {
                    final String[] splitted = node.split(NODE_SPLITTER);
                    final String host = splitted[0];
                    final int port = Integer.parseInt(splitted[1]);
                    redisSentinelConfiguration.addSentinel(RedisNode.newRedisNode()
                            .listeningAt(host, port)
                            .build());
                });
    }

    @Bean
    public LettucePoolingClientConfiguration lettucePoolingClientConfiguration(ClientOptions clientOptions, ClientResources clientResources, RedisProperties redisProperties) {
        return LettucePoolingClientConfiguration.builder()
                .readFrom(ReadFrom.ANY_REPLICA)
                .poolConfig(genericObjectPoolConfig(redisProperties))
                .clientOptions(clientOptions)
                .clientResources(clientResources)
                .build();
    }

    @Bean
    public GenericObjectPoolConfig genericObjectPoolConfig(RedisProperties redisProperties) {
        final GenericObjectPoolConfig config = new GenericObjectPoolConfig();
        config.setMaxIdle(redisProperties.getPoolMaxIdle());
        config.setMinIdle(redisProperties.getPoolMinIdle());
        config.setMaxTotal(redisProperties.getPoolMaxTotal());
        config.setBlockWhenExhausted(false);
        config.setMaxWaitMillis(redisProperties.getPoolMaxWaitMillis());
        return config;
    }

    @Bean
    public ClientOptions clientOptions(RedisProperties redisProperties) {
        return ClientOptions.builder()
                .autoReconnect(true)
                .disconnectedBehavior(ClientOptions.DisconnectedBehavior.REJECT_COMMANDS)
                .timeoutOptions(TimeoutOptions.builder().fixedTimeout(Duration.ofSeconds(redisProperties.getCommandTimedOutSec())).build())
                .build();
    }

    @Bean(destroyMethod = "shutdown")
    public ClientResources clientResources() {
        return DefaultClientResources.create();
    }

    you should import new lettuce core version for load balance read mode(ReadFrom.ANY_REPLICA), it will come with spring-boot 2.4.0

    pom.xml

            <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>io.lettuce</groupId>
                    <artifactId>lettuce-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>io.lettuce</groupId>
            <artifactId>lettuce-core</artifactId>
            <version>6.0.1.RELEASE</version>
    </dependency>
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search