Azure Question posted in
The official documentation to get you started can be found here.

Is there any efficient way to download selected secrets from a common Azure Key Vault of an organization?

Our Azure keyvault instance is platform specific which means it will have tons & tons secrets belong to various app in the platform.

In our business requirement we need to download the secrets which are specific to our individual app from that common platform specific keyvault.

what would be the best and efficient way to do the above requirement? pagination is something can be considered but there shouldn’t be any performance bottleneck or anti-pattern.

//code to get all secrets from that keyvault 
public async IAsyncEnumerable<string> GetKeys()
{
    var keyVaultSecureValueManager = (KeyVaultSecureValueManager)this.SecureValueManager;
    var allPages = keyVaultSecureValueManager.Client.GetPropertiesOfSecretsAsync().AsPages();

    await foreach (var page in allPages)
    {
        foreach (var secretProperties in page.Values)
        {
            yield return secretProperties.Name;
        }
    }
}

//filter locally, the required keys alone.
 private static async Task<List<string>> GetKeysAsync(IAsyncEnumerable<string> keyvaultKeys)
    {
        List<string> keys = new List<string>();
        await foreach (var key in keyvaultKeys)
        {
            if (key.StartsWith("myKeys", StringComparison.InvariantCultureIgnoreCase))
            {
                keys.Add(key);
            }
        }

        return keys;
    }

 //get individual secret
 private async Task<KeyVaultResponse> GetCertAsync(string key)
    {
        return await secretManager.GetKeyAsync(key);
    }


 //code get secret value of selected secrets from the all secrets properties
 public async Task<Certs> GetAllCertsAsync()
    {
        var keyvaultKeys = this._secretManager.GetKeys();
        var selectedKeys = await GetKeysAsync(keyvaultKeys);
        var allCertKeys = selectedKeys.Where(s => s.EndsWith("cert", StringComparison.InvariantCultureIgnoreCase));
        var allPwdKeys = selectedKeys.Where(s => s.EndsWith("pwd", StringComparison.InvariantCultureIgnoreCase));

        // Fetch all certificates concurrently
        var fetchTasks = allCertKeys.Select(c => this.GetCertAsync(c, allPwdKeys));
        var results = await Task.WhenAll(fetchTasks);

        // Check for any bad requests
        var badRequest = results.FirstOrDefault(res => res.IsBadRequest);
        if (badRequest != null)
        {
            return BadRequestResponse(badRequest.ErrorMessage);
        }

        return results;
    }

The above code lives inside Web API microservices & deployed on AKS. No Azure functions/Web Apps or any other kind app. entertained in our current ecosystem.

As per our analysis async IAsyncEnumerable<string> GetKeys() makes hundreds of roundtrips to the server as it has to download all those tons of secrets properties from server. Any way you we can avoid those many unwanted round trips? by doing server side filter using any Azure key vault SDKs?

Any recommended practice to group the secrets per app specific or domain specific with in the common Azure key vault instance, so just to query that collection or group alone to improve efficiency?

We use SecretClient Class under namespace Azure.Security.KeyVault.Secrets.
The above code has performance bottle neck. Any better alternatives or how the above code can be made more efficient without any anti-patterns?

Tags:

2

Answers


  1. 0

    If the goal is just to organize secrets in a key vault, you can use tags (with application names, for example). However, this does not help solve the performance bottleneck, as the Key Vault’s REST Api doesn’t support filtering by tags.

    I recommend considering App Configuration in conjunction with the Key Vault reference feature. App Configuration supports labels (similar to Key Vault’s tags) and supports filtering by labels. In other words, your app can read only secrets marked with a specific label(-s) and/or unlabelled secrets.

    builder.Configuration.AddAzureAppConfiguration(options =>
    {
            options.Connect(builder.Configuration.GetConnectionString("AppConfig"))
                .Select(KeyFilter.Any, "app1")
                .Select(KeyFilter.Any, LabelFilter.Null);
    });

    The huge benefit of using App Configuration is its C# middleware that implements caching under the hood and would solve your performance concerns.

    Let me emphasize that your secrets will remain in the key vault. App Configuration will be only a proxy between the consuming app and the Key Vault (see a picture below).

    enter image description here

    Login or Signup to reply.
  2. 0

    Any recommended practice to group the secrets per app specific or domain specific with in the common Azure key vault instance, so just to query that collection or group alone to improve efficiency?

    You can use a prefix for your secrets based on the app or domain, e.g. myapp_cosmosdb_connection_string or mydomain_redis_connection_string. Or, as an alternative, use tags as recommended by Artem.

    As per our analysis async IAsyncEnumerable GetKeys() makes hundreds of roundtrips to the server as it has to download all those tons of secrets properties from server. Any way you we can avoid those many unwanted round trips? by doing server side filter using any Azure key vault SDKs?

    I’m not sure if there is any sort of "server-side filtering", but as a starting point and to keep it simple maybe try reading all secrets from the keyvault and use in-memory cache to avoid the round trips and consequently other issues such as throttling limits?

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search