Memcached – Update In-memory cache for every instance in AppEngine (automatic scaling)

Accessing all dynamic instances is usually troublesome and something you should not rely on.

Instead redesign and use a different approach.

Have all your instances use an in-memory cache, but use an expiration time for the cached data. Whenever such data is needed, first check if the data is still valid (check the expiration time), and if it is, go ahead and use it. If it has expired, fetch the new, actual data from “some” place. This “some” place may be the Memcache or the Datastore, or optionally both like first try in the Memcache, if not there, then from Datastore; or it may be at a completely different place, even outside of Google Cloud Platform. Fetching new data should contain its expiration time.

This approach does not require you to reach the dynamic instances, they will take care of refreshing their cached data once it expires, automatically.

If accessed from multiple goroutines, access to the cached data must be synchronized. Best would be to use a sync.RWMutex, so you can allow multiple readers without blocking each other (frequent operation), and only acquire write lock if the cached data has expired and needs to be refreshed.

Here’s an example implementation of such in-memory cache:

func getFreshData() (data interface{}, expires time.Time, err error) {
    // Implement getting fresh data here:
    return nil, time.Now().Add(time.Minute), nil
}

type cachedData struct {
    sync.RWMutex
    data    interface{}
    expires time.Time
}

var cd = new(cachedData) // zero value is ready to use

func Get() (data interface{}, err error) {
    cd.RLock()
    if time.Now().Before(cd.expires) {
        // We're done: we can use the cached data:
        data = cd.data
        cd.RUnlock()
        return
    }

    cd.RUnlock()

    // Either we don't have cached data or it has expired.
    // Acquire write lock and get data
    cd.Lock()
    defer cd.Unlock()

    // But once we have the write lock, check again, as another competing
    // goroutine might have fetched data before us:
    if time.Now().Before(cd.expires) {
        // Another goroutine fetched fresh data:
        return cd.data, nil
    }

    // Nope, we have to do it ourselves:
    data, expires, err = getFreshData()
    if err == nil {
        // Also put fresh data into the cache:
        cd.data = data
        cd.expires = expires
    } else {
        // There was an error getting it, set a 5 sec timeout to not keep calling:
        cachedData.data = nil
        cachedData.expires = time.Now().Add(5 * time.Second)
    }

    return
}