Asp.net – "IDX10500: Signature validation failed." error with self-hosted KeyCloak and API Gateway Ocelot

So as stated in the title, I am having issues with the token validation in dotnet application. Here is the program.cs code:

using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.IdentityModel.Tokens;
using Ocelot.DependencyInjection;
using Ocelot.Middleware;

var builder = WebApplication.CreateBuilder(args);

// Add configuration for Ocelot
builder.Configuration.AddJsonFile("ocelot.json", optional: false, reloadOnChange: true);

// Add Ocelot service
builder.Services.AddOcelot(builder.Configuration);

builder.Services.AddHttpClient();
// Keycloak configuration
builder.Services
    .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer("KeyCloak", options =>
    {
        options.Authority = "http://localhost:5002/realms/test-realm";
        options.Audience = "api-gw";
        options.MetadataAddress = "http://keycloak:5002/realms/test-realm/.well-known/openid-configuration";
        options.RequireHttpsMetadata = false;
        
        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidateIssuer = true,
            ValidIssuer = "http://localhost:5002/realms/test-realm",
            ValidateAudience = true,
            ValidAudience = "api-gw",
            ValidateLifetime = true,
            ValidateIssuerSigningKey = true,
            ClockSkew = TimeSpan.Zero
        };
    

        options.Events = new JwtBearerEvents
        {
            OnTokenValidated = context =>
            {
                Console.WriteLine("Token Validated.");
                return Task.CompletedTask;
            },
            OnAuthenticationFailed = context => 
            { 
                Console.WriteLine($"Token Validation Failed: {context.Exception.Message}");
                return Task.CompletedTask;
            }
        };
    });

// Add authorization
builder.Services.AddAuthorization();

var app = builder.Build();

// Use authentication middleware before Ocelot
app.UseAuthentication();
app.UseAuthorization();
app.UseHttpsRedirection();

// Use Ocelot API Gateway
app.UseOcelot().Wait();

// Run the app
app.Run();

What I though would happen is that the Microsoft.AspNetCore.Authentication.JwtBearer would automatically fetch the public keys from the KeyCloak URL from MetadataAddress. Both the KeyCloak and the API Gateways are deployed as Docker containers. I have checked the connectivity, the containers can communicate. Additionally, when checking in local browser the http://localhost:5002/realms/test-realm/.well-known/openid-configuration I am receiving the proper JSON object with a link to the jwks_uri. However, I am getting the following error:

info: Ocelot.Authentication.Middleware.AuthenticationMiddleware[0]
2024-12-16 20:44:49       requestId: 0HN8U187GSHNG:00000003, previousRequestId: No PreviousRequestId, message: 'The path '/api/weatherforecast' is an authenticated route! AuthenticationMiddleware checking if client is authenticated...'
2024-12-16 20:44:50 Token Validation Failed: IDX10500: Signature validation failed. No security keys were provided to validate the signature.
2024-12-16 20:44:50 warn: Ocelot.Authentication.Middleware.AuthenticationMiddleware[0]
2024-12-16 20:44:50       requestId: 0HN8U187GSHNG:00000003, previousRequestId: No PreviousRequestId, message: 'Client has NOT been authenticated for path '/api/weatherforecast' and pipeline error set. Request for authenticated route '/api/weatherforecast' was unauthenticated;'
2024-12-16 20:44:50 warn: Ocelot.Responder.Middleware.ResponderMiddleware[0]
2024-12-16 20:44:50       requestId: 0HN8U187GSHNG:00000003, previousRequestId: No PreviousRequestId, message: 'Error Code: UnauthenticatedError Message: Request for authenticated route '/api/weatherforecast' was unauthenticated errors found in ResponderMiddleware. Setting error response for request path:/api/weatherforecast, request method: GET'

For the sake of completeness, here is the ocelot.json. The /login route goes through another small auth-service container, but I am able to receive the token just fine. The problem is with the authentication for the /weatherforecast route.

{
  "Routes": [
    {
      "DownstreamPathTemplate": "/login",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "auth-service",
          "Port": 8080
        }
      ],
      "UpstreamPathTemplate": "/login",
      "UpstreamHttpMethod": ["Post"]
    },
    {
      "DownstreamPathTemplate": "/todos/{id}",
      "DownstreamScheme": "https",
      "DownstreamHostAndPorts": [
        {
          "Host": "jsonplaceholder.typicode.com",
          "Port": 443
        }
      ],
      "UpstreamPathTemplate": "/api/todos/{id}",
      "UpstreamHttpMethod": ["Get"]
    },
    {
      "DownstreamPathTemplate": "/weatherforecast",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "temp-service",
          "Port": 8080
        }
      ],
      "UpstreamPathTemplate": "/api/weatherforecast",
      "UpstreamHttpMethod": ["Get"],
      "AuthenticationOptions": {
        "AuthenticationProviderKey": "KeyCloak",
        "AllowedScopes": []
      }
    }
  ],
  "GlobalConfiguration": {
    "BaseUrl": "https://localhost:5000"
  }
}

Any ideas on what could be causing the problem? From what I can deduct, the keys are not correctly loaded from the KeyCloak URL, thus the JWT library cannot validate the token without any public key.