Been having some problems with a web based .Net(C#) application. I’m using the LazyCache library to cache frequent JSON responses (some in & around 80+KB) for users belonging to the same company across user sessions.
One of the things we need to do is to keep track of the cache keys for a particular company so when any user in the company makes mutating changes to items being cached we need to clear the cache for those items for that particular company’s users to force the cache to be repopulated immediately upon the receiving the next request.
We choose LazyCache library as we wanted to do this in memory without needing to use an external cache source such as Redis etc as we don’t have heavy usage.
One of the problems we have using this approach is we need to keep track of all the cache keys belonging to a particular customer anytime we cache. So when any mutating change is made by company user’s to the relevant resource we need to expire all the cache keys belonging to that company.
To achieve this we have a global cache which all web controllers have access to.
private readonly IAppCache _cache = new CachingService();
protected IAppCache GetCache()
{
return _cache;
}
A simplified example (forgive any typos!) of our controllers which use this cache would be something like below
[HttpGet]
[Route("{customerId}/accounts/users")]
public async Task<Users> GetUsers([Required]string customerId)
{
var usersBusinessLogic = await _provider.GetUsersBusinessLogic(customerId)
var newCacheKey= "GetUsers." + customerId;
CacheUtil.StoreCacheKey(customerId,newCacheKey)
return await GetCache().GetOrAddAsync(newCacheKey, () => usersBusinessLogic.GetUsers(), DateTimeOffset.Now.AddMinutes(10));
}
We use a util class with static methods and a static concurrent dictionary to store the cache keys – each company (GUID) can have many cache keys.
private static readonly ConcurrentDictionary<Guid, ConcurrentHashSet<string>> cacheKeys = new ConcurrentDictionary<Guid, ConcurrentHashSet<string>>();
public static void StoreCacheKey(Guid customerId, string newCacheKey)
{
cacheKeys.AddOrUpdate(customerId, new ConcurrentHashSet<string>() { newCacheKey }, (key, existingCacheKeys) =>
{
existingCacheKeys.Add(newCacheKey);
return existingCacheKeys;
});
}
Within that same util class when we need to remove all cache keys for a particular company we have a method similar to below (which is caused when mutating changes are made in other controllers)
public static void ClearCustomerCache(IAppCache cache, Guid customerId)
{
var customerCacheKeys = new ConcurrentHashSet<string>();
if (!cacheKeys.TryGetValue(customerId,out customerCacheKeys))
{
return new ConcurrentHashSet<string>();
}
foreach (var cacheKey in customerCacheKeys)
{
cache.Remove(cacheKey);
}
cacheKeys.TryRemove(customerId, out _);
}
We have recently been getting performance problems that our web requests response time slow significantly over time – we don’t see significant change in terms of the number of requests per second.
Looking at the garbage collection metrics we seem to notice a large Gen 2 heap size and a large object size which seem to going upwards – we don’t see memory been reclaimed.
We are still in the middle of debugging this but I’m wondering could using the approach described above lead to the problems we are seeing. We want thread safety but could there be an issue using the concurrent dictionary we have above that even after we remove items that memory is not being freed leading to excessive Gen 2 collection.
Also we are using workstation garbage collection mode, imagine switching to server mode GC will help us (our IIS server has 8 processors + 16 GBs ram) but not sure switching will fix all the problems.
2
Answers
Large objects (> 85k) belong in gen 2 Large Object Heap (LOH), and they are pinned in memory.
No compaction, but only reallocation may lead to memory fragmentation.
Long running server processes can be done in by this – it is not uncommon.
You are probably seeing fragmentation occur over time.
Server GC just happens to be multi-threaded – I wouldn’t expect it to solve fragmentation.
You could try breaking up your large objects – this might not be feasible for your application.
You can try setting
LargeObjectHeapCompaction
after a cache clear – assuming it’s infrequent.Ultimately, I’d suggest profiling the heap to find out what works.
You may want to take advantage of the
ExpirationTokens
property of theMemoryCacheEntryOptions
class. You can also use it from theICacheEntry
argument passed in the delegate of theLazyCache.Providers.MemoryCacheProvider.GetOrCreateAsync
method. For example:Now the
GetCustomerExpirationToken
should return an object implementing theIChangeToken
interface. Things are becoming a bit complex, but bear with me for a minute. The .NET platform doesn’t provide a built-inIChangeToken
implementation suitable for this case, since it is mainly focused on file system watchers. Implementing one is not difficult though:This is a general implementation of the
IChangeToken
interface, that is activated manually with theSignalChanged
method. The signal will be propagated to the underlyingMemoryCache
object, which will subsequently invalidate all entries associated with this token.Now what is left to do is to associate these tokens with a customer, and store them somewhere. I think that a
ConcurrentDictionary
should be quite adequate:Finally the method that is needed to signal that all entries of a specific customer should be invalidated: