Here is my situation — my development team must get its connection strings and other sensitive values like passwords from machine-level environment variables on Windows servers. I have a .NET 6 Console App using Generic Host, and in my appsettings.json file, I store the name of the environment variable I want to retrieve the value for, and I have a class that holds the environment variable name and the actual secret value that gets obtained from the environment variable.
My problem is that after I get the value from the environment variable and store it in TopSecretConfig.SecretValue, the value is accessible from from IOptions on my machine, whether I’m in Visual Studio 2022 or running the application from the command line. However, when it’s deployed to our QA environment, the value is null when I access it from IOptions.
Obviously, the code works (on my machine), but is that the proper way to alter the configuration object and have the values bound so they are properly injected via IOptions later?
What environmental factors would prevent the code from working in another environment? The environment variables are present in the QA environment at the machine level.
The code below is not my actual code, but very close.
appsettings.json:
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.Hosting.Lifetime": "Information"
}
},
"TopSecretConfig": {
"EnvironmentVariableName": "VARIABLE_NAME"
}
}
Class to hold config value:
public class TopSecretConfig
{
public string EnvironmentVariableName { get; set; }
public string SecretValue { get; set; }
}
Generic host snippet to get config:
using (var host = Host.CreateDefaultBuilder(args)
.ConfigureLogging((context, builder) =>
{
// logging config
})
.ConfigureAppConfiguration((context, appConfig) =>
{
appConfig.Sources.Clear();
appConfig.SetBasePath(context.HostingEnvironment.ContentRootPath);
appConfig.AddJsonFile("appSettings.json");
appConfig.AddEnvironmentVariables();
appConfig.AddEnvironmentVariables("DOTNET_");
appConfig.AddEnvironmentVariables("TOP_SECRET_");
})
.ConfigureServices((builder, services) =>
{
services.Configure<TopSecretConfig>(builder.Configuration.GetSection(nameof(TopSecretConfig)));
// --------- Is this allowed? -----------
services.Configure<TopSecretConfig>(config =>
{
config.SecretValue = builder.Configuration[config.EnvironmentVariableName];
});
services.AddSingleton<Processor>();
})
.Build())
{
await host.StartAsync();
var lifetime = host.Services.GetRequiredService<IHostApplicationLifetime>();
var processor = host.Services.GetRequiredService<Processor>();
await processor.ExecuteAsync();
lifetime.StopApplication();
await host.WaitForShutdownAsync();
}
Class that uses config:
public class Processor
{
private readonly ILogger<Processor> Logger;
private readonly TopSecretConfig SecretConfig;
public Processor(ILogger<TopSecretProcessor> logger, IOptions<TopSecretConfig> secretConfig)
{
this.Logger = logger ?? throw new ArgumentNullException(nameof(logger));
this.SecretConfig = secretConfig?.Value ?? throw new ArgumentNullException(nameof(secretConfig));
// this.SecretConfig.SecretValue is null here.
}
}
2
Answers
If the variable has the same name as the one in your config it will automatically be overlayed onto the config file object. Usually that is how its done. That way it doesn’t matter if its an environment variable config value or something thats pulled from a cloud secret service like azure keyvault.
Then you use a colon (or __ if your environment doesn’t support colon) to create the json hierarchy so for your json file example it would be
TopSecretConfig:EnviornmentVariableName
or
TopSecretConfig__EnviornmentVariableName
I would also validate that your config is not null somewhere outside of your processor using Validate or ValidateOnStart methods on the config probably in program.cs. That way your app fails right away instead of potentially looking like it started correctly and then when the processor gets executed it unexpectedly errors out.
Also not sure if this is intentional but in your TopSecretConfig object. I’m not sure why you have EnvironmentVariableName in there as a variable with a separate SecretValue variable. When you read in the configs (from both the json and environment variables) the secret would get stored under EnvironmentVariableName as it’s string value and SecretValue would be unused.
The configuration system in [ASP].NET Core, Microsoft.Extensions.Configuration (M.E.C) is a vast improvement over the one in "Classic" [ASP].NET, using App.config/Web.config.
One of the benefits is that configuration can be layered. The layers are merged into a unified view, so the source of a value is unimportant. What’s important is that it’s available in one place regardless of how it got there.
There was no actual configuration in the question, so I’ll use simple SMTP settings as an example since it contains elements that should be sourced in a variety of ways.
Starting with
appsettings.json
, it should have everything your production deployment needs, minus the truly secret things. For the truly secret things, I usually have commented-out properties to show that they would be there once everything is merged. This makesappsettings.json
a reliable source of information about the entire configuration, even if some of it is supplied elsewhere.For this example, the SMTP settings will include the host, port, user name, and password.
The
"Smtp"
property can be bound the an instance of the following class:To enable injection as
IOptions<SmtpSettings>
, you can add it to the IoC container (M.E.DI) from theEmail:Smtp
section like this:In the development environment, the host and user name might be different. Specify just this much of the configuration and these values will replace the ones in
appsettings.json
while everything else, like the port, remains unaffected.Each developer’s user secrets could contain things that fall into two categories:
Something I’ve been doing to help developers onboard when joining a project is to put a
usersecrets.sample.json
file in the root of the project. It shows everything they need to configure insecrets.json
to make the application work when debugging. This way, they don’t have to find out later and have to track someone down.The default application builders add environment variables after the JSON files, and command line parameters after that.
So the default application builder layers the configuration like this:
As sources are stacked, those values are overlaid onto the others, and merged downward (as visualized above).
Of the things that go in a developer’s user secrets, only those in the first category, things that can’t go in included in source control, also need to be configured in the deployed environment. The application will likely be running as the AppPool so configuring user secrets isn’t going to work (*well, maybe you can do it, but it’s not a good idea). This is where environment variables are useful.
To configure the SMTP password using an environment variable, use the colon-separated path of JSON properties, such as
Email:Smtp:Password
for the example above. When binding theEmail:Smtp
section to anSmtpSettings
instance, it will see that as aPassword
property in that section and bind it just as if it had been in the JSON.When publishing, you can add the
<EnvironmentName>
to the publish profile (.pubxml
file) and it will include the elements needed inweb.config
set theASPDOTNET_ENVIRONMENT
environment variable. However, this only works for that one environment variable, not for arbitrary environment variables. If you need others, you have to configure them another way.In IIS, you can set environment variables in
applicationhost.config
, which means that they won’t be inweb.config
and they won’t be overwritten when you deploy. I won’t get into the details because there’s an answer on StackOverflow that does a better job: Publish to IIS, setting Environment VariableIf you’re deploying a Windows Service, environment variables are set in a registry value under the service’s registry key. After creating the service, add the
Environment
value to the key. See this Server Fault post for details about how to do that: https://serverfault.com/questions/813506/setting-environment-variable-for-service