skip to Main Content

Been having some problems with a web based .Net(C#) application. I’m using the LazyCache library to cache frequent JSON responses (some in & around 80+KB) for users belonging to the same company across user sessions.

One of the things we need to do is to keep track of the cache keys for a particular company so when any user in the company makes mutating changes to items being cached we need to clear the cache for those items for that particular company’s users to force the cache to be repopulated immediately upon the receiving the next request.

We choose LazyCache library as we wanted to do this in memory without needing to use an external cache source such as Redis etc as we don’t have heavy usage.

One of the problems we have using this approach is we need to keep track of all the cache keys belonging to a particular customer anytime we cache. So when any mutating change is made by company user’s to the relevant resource we need to expire all the cache keys belonging to that company.

To achieve this we have a global cache which all web controllers have access to.

private readonly IAppCache _cache = new CachingService();

protected IAppCache GetCache()
{
    return _cache;
}

A simplified example (forgive any typos!) of our controllers which use this cache would be something like below

[HttpGet]
[Route("{customerId}/accounts/users")]
public async Task<Users> GetUsers([Required]string customerId)
{
    var usersBusinessLogic = await _provider.GetUsersBusinessLogic(customerId)

    var newCacheKey= "GetUsers." + customerId;

    CacheUtil.StoreCacheKey(customerId,newCacheKey)

    return await GetCache().GetOrAddAsync(newCacheKey, () => usersBusinessLogic.GetUsers(), DateTimeOffset.Now.AddMinutes(10));
}

We use a util class with static methods and a static concurrent dictionary to store the cache keys – each company (GUID) can have many cache keys.

private static readonly ConcurrentDictionary<Guid, ConcurrentHashSet<string>> cacheKeys = new ConcurrentDictionary<Guid, ConcurrentHashSet<string>>();

public static void StoreCacheKey(Guid customerId, string newCacheKey)
{
    cacheKeys.AddOrUpdate(customerId, new ConcurrentHashSet<string>() { newCacheKey }, (key, existingCacheKeys) =>
    {
        existingCacheKeys.Add(newCacheKey);
        return existingCacheKeys;
    });
}

Within that same util class when we need to remove all cache keys for a particular company we have a method similar to below (which is caused when mutating changes are made in other controllers)

public static void ClearCustomerCache(IAppCache cache, Guid customerId)
{
    var customerCacheKeys = new ConcurrentHashSet<string>();

    if (!cacheKeys.TryGetValue(customerId,out customerCacheKeys))
    {
        return new ConcurrentHashSet<string>();
    }


    foreach (var cacheKey in customerCacheKeys)
    {
        cache.Remove(cacheKey);
    }

    cacheKeys.TryRemove(customerId, out _);
}

We have recently been getting performance problems that our web requests response time slow significantly over time – we don’t see significant change in terms of the number of requests per second.

Looking at the garbage collection metrics we seem to notice a large Gen 2 heap size and a large object size which seem to going upwards – we don’t see memory been reclaimed.

We are still in the middle of debugging this but I’m wondering could using the approach described above lead to the problems we are seeing. We want thread safety but could there be an issue using the concurrent dictionary we have above that even after we remove items that memory is not being freed leading to excessive Gen 2 collection.

Also we are using workstation garbage collection mode, imagine switching to server mode GC will help us (our IIS server has 8 processors + 16 GBs ram) but not sure switching will fix all the problems.

2

Answers


  1. Large objects (> 85k) belong in gen 2 Large Object Heap (LOH), and they are pinned in memory.

    1. GC scans LOH and marks dead objects
    2. Adjacent dead objects are combined into free memory
    3. The LOH is not compacted
    4. Further allocations only try to fill in the holes left by dead objects.

    gc LOH

    No compaction, but only reallocation may lead to memory fragmentation.
    Long running server processes can be done in by this – it is not uncommon.
    You are probably seeing fragmentation occur over time.

    Server GC just happens to be multi-threaded – I wouldn’t expect it to solve fragmentation.

    You could try breaking up your large objects – this might not be feasible for your application.

    You can try setting LargeObjectHeapCompaction after a cache clear – assuming it’s infrequent.

    GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;
    GC.Collect();
    

    Ultimately, I’d suggest profiling the heap to find out what works.

    Login or Signup to reply.
  2. You may want to take advantage of the ExpirationTokens property of the MemoryCacheEntryOptions class. You can also use it from the ICacheEntry argument passed in the delegate of the LazyCache.Providers.MemoryCacheProvider.GetOrCreateAsync method. For example:

    Task<T> GetOrAddAsync<T>(string key, Func<Task<T>> factory,
        int durationMilliseconds = Timeout.Infinite, string customerId = null)
    {
        return GetMemoryCacheProvider().GetOrCreateAsync<T>(key, (options) =>
        {
            if (durationMilliseconds != Timeout.Infinite)
            {
                options.SetSlidingExpiration(TimeSpan.FromMilliseconds(durationMilliseconds));
            }
            if (customerId != null)
            {
                options.ExpirationTokens.Add(GetCustomerExpirationToken(customerId));
            }
            return factory();
        });
    }
    

    Now the GetCustomerExpirationToken should return an object implementing the IChangeToken interface. Things are becoming a bit complex, but bear with me for a minute. The .NET platform doesn’t provide a built-in IChangeToken implementation suitable for this case, since it is mainly focused on file system watchers. Implementing one is not difficult though:

    class ChangeToken : IChangeToken, IDisposable
    {
        private volatile bool _hasChanged;
        private readonly ConcurrentQueue<(Action<object>, object)>
            registeredCallbacks = new ConcurrentQueue<(Action<object>, object)>();
    
        public void SignalChanged()
        {
            _hasChanged = true;
            while (registeredCallbacks.TryDequeue(out var entry))
            {
                var (callback, state) = entry;
                callback?.Invoke(state);
            }
        }
    
        bool IChangeToken.HasChanged => _hasChanged;
    
        bool IChangeToken.ActiveChangeCallbacks => true;
    
        IDisposable IChangeToken.RegisterChangeCallback(Action<object> callback,
            object state)
        {
            registeredCallbacks.Enqueue((callback, state));
            return this; // return null doesn't work
        }
    
        void IDisposable.Dispose() { } // It is called by the framework after each callback
    }
    

    This is a general implementation of the IChangeToken interface, that is activated manually with the SignalChanged method. The signal will be propagated to the underlying MemoryCache object, which will subsequently invalidate all entries associated with this token.

    Now what is left to do is to associate these tokens with a customer, and store them somewhere. I think that a ConcurrentDictionary should be quite adequate:

    private static readonly ConcurrentDictionary<string, ChangeToken>
        CustomerChangeTokens = new ConcurrentDictionary<string, ChangeToken>();
    
    private static ChangeToken GetCustomerExpirationToken(string customerId)
    {
        return CustomerChangeTokens.GetOrAdd(customerId, _ => new ChangeToken());
    }
    

    Finally the method that is needed to signal that all entries of a specific customer should be invalidated:

    public static void SignalCustomerChanged(string customerId)
    {
        if (CustomerChangeTokens.TryRemove(customerId, out var changeToken))
        {
            changeToken.SignalChanged();
        }
    }
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search