skip to Main Content

We are using Cloud Memorystore Redis instance to add a caching layer to our mission critical Internet facing application. Total number of calls (including get, set and key expiry operations) to Memorystore instance is around 10-15K per second. CPU utilisation has been consistently around 75-80% and expecting the utilisation to go even higher.

Currently, we are using M4 capacity tier under Standard service tier.

https://cloud.google.com/memorystore/docs/redis/pricing

Need some clarity around the following pointers.

  1. How many CPU cores do the M4 capacity tier correspond to?
  2. Is it really alarming to have more than 100% CPU utilisation? Do we expect any noticeable performance issues?
  3. What are the options to tackle the performance issues (if any) caused by higher CPU utilisation (>=100%)? Will switching to M5 capacity tier address the high CPU consumption and the corresponding issues.

Our application is really CPU intensive and we don’t see any way to further optimize our application. Looking forward to some helpful references.

2

Answers


  1. Addressing your questions.

    1. How many CPU cores do the M4 capacity tier correspond to?
    Cloud Memorystore for Redis is a Google-managed service which means that Google can reserve the inner details(resources) of the virtual machine that is running the redis service. Still it is expected that the higher the capacity tier, the more resources(CPU) the virtual machine will have. For your case in particular, adding CPUs will not solve issues around CPU usage because redis service itself is single threaded.

    As you can see from the previous link:

    To maximize CPU usage you can start multiple instances of Redis.
    If you want to use multiple CPUs you can start thinking of some way to shard earlier.

    2. Is it really alarming to have more than 100% CPU utilisation?
    Yes, it is alarming to have high CPU utilization because it can result in connection errors or high latency.
    CPU utilization is important but also whether the Redis instance is efficient enough to sustain your throughput at a given latency. You can check the redis latency with the command redis-cli --latency while CPU % is high.

    3. Do we expect any noticeable performance issues?
    This is really hard to say or predict because it depends on several factor(client service, commands run within a time frame, workload). Some of the most common causes for high latency and performance issues are:

    • Client VMs or services are overloaded and not consuming the messages from Redis: When a client opens a TCP connection to redis then the redis server has a buffer of messages to send to that connection. If a client service has its CPU maxed out, giving no time for the kernel to receive messages from redis then they fill up on the redis server.

    • The commands executed are consuming a lot of CPU: The following commands are known to be potentially very expensive to process:
      EVAL/EVALSHA
      KEYS
      LRANGE
      ZRANGE/ZREVRANGE

    4.-What are the options to tackle the performance issues (if any) caused by higher CPU utilisation (>=100%)?
    This question revolves mainly around the scaling design of your implementation. Since redis is single threaded, a better approach to reduce CPU % would be by sharding your data in multiple redis instances and have a proxy in front of it to distribute the load. Please take a look at the graph under section Twemproxy from this link.

    5.-Will switching to M5 capacity tier address the high CPU consumption and the corresponding issues?
    Switching to a higher capacity tier should help with the latency temporarily but this is known as vertical scaling which is limited to the tiers that Cloud Memorystore offers.

    Login or Signup to reply.
  2. Redis Enterprise solves all the issues you are facing. Redis Enterprise can be configured in a clustered configuration and utilize all the resources of the machine as well as scale out over multiple machines.

    The Redis Enterprise Software is responsible for watching over the CPU utilization and other resource management tasks so you do not need to.
    It is offered on GCP and GCP marketplace as well.

    https://redis.com/redis-enterprise-cloud/pricing/

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search