Question posted in 
I’ve deliberately remove sensitive information in the codes and image
I’m facing couple of issues with setting up using Docker and Nginx. My solutions/service have been dockizer successfully but the services are unable to communicate via the Nginx.
I’ve a docker that contains 3 services which are my solutions.
Auth Service; to authentication user when logging in using OpenID ConnectPortal Service; authenticated user to access various viewsAPI Service; an API to Get or Post based on user request.
Also to note Nginx is also a service in my Docker. And all my services the ports are expose correctly
Details of the set up as follows:
Auth Serviceis in a reverse proxy with server name as id-local..comPortal Serviceis in a reverse proxy with the server name as app-
local..comAPI Serviceis an API gateway share the same server name as Portal Service; app-local..com- Both servers are running with SSL and HTTP2; I’ve omit out SSL related cert.
- Both servers are actually the same IP address as they are sit on the same PC. Do note that the port number in the
Nginxmight be wrong as I was following a tutorial.
Issue 1 User has log in successfully via Auth Service and Auth Service shall re-direct to my Portal Service via Nginx. But instead I encounter an issue whereby Portal Service is unable to get the Auth Service‘s Discovery EndPoint.
It is also to note that Auth Service; once user log in successfully, the service shall generate two cookies; idsrv.session and an external .net cookies. In IIS, those two cookies are to be shared throughout the services. The Portal Service, requires the metadata of Auth Service in order to generate a JWT cookies.
Error log:
Request starting HTTP/1.1 GET http://app-local..com:44343/favicon.ico - -
Executing endpoint '..Web.Controllers.ErrorController.Route (..Web)'
Route matched with {action = "Route", controller = "Error"}. Executing controller action with signature Microsoft.AspNetCore.Mvc.IActionResult Route(System.String) on controller ..Web.Controllers.ErrorController (..Web).
Authorization failed. These requirements were not met:
DenyAnonymousAuthorizationRequirement: Requires an authenticated user.
[INF] Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
[INF] Executing ChallengeResult with authentication schemes ([]).
[INF] Executed action ..Web.Controllers.ErrorController.Route (.Web) in 3.2872ms
[INF] Executed endpoint '..Web.Controllers.ErrorController.Route (.Web)'
[ERR] An unhandled exception has occurred while executing the request.System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://id-local..com:5200/.well-known/openid-configuration'.
---> System.IO.IOException: IDX20804: Unable to retrieve document from: 'https://id-local..com:5200/.well-known/openid-configuration'.
---> System.Net.Http.HttpRequestException: Connection refused (id-local..com:5200)
---> System.Net.Sockets.SocketException (111): Connection refused
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|283_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
--- End of inner exception stack trace ---
at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(String address, IDocumentRetriever retriever, CancellationToken cancel)
at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)
--- End of inner exception stack trace ---
at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)
at Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.HandleChallengeAsyncInternal(AuthenticationProperties properties)
at Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.HandleChallengeAsync(AuthenticationProperties properties)
at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.ChallengeAsync(AuthenticationProperties properties)
at Microsoft.AspNetCore.Authentication.AuthenticationService.ChallengeAsync(HttpContext context, String scheme, AuthenticationProperties properties)
at Microsoft.AspNetCore.Mvc.ChallengeResult.ExecuteResultAsync(ActionContext context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAlwaysRunResultFilters>g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at WebMarkupMin.AspNetCore3.WebMarkupMinMiddleware.ProcessAsync(HttpContext context, Boolean useMinification, Boolean useCompression)
at WebMarkupMin.AspNetCore3.WebMarkupMinMiddleware.ProcessAsync(HttpContext context, Boolean useMinification, Boolean useCompression)
at WebMarkupMin.AspNetCore3.WebMarkupMinMiddlewareBase.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.StatusCodePagesExtensions.<>c__DisplayClass6_0.<<UseStatusCodePagesWithReExecute>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)366
[INF] Request finished HTTP/1.1 GET http://app-local..com:44343/favicon.ico - - - 500 - text/plain 8.9ms
Part of Nginx Config file
upstream auth
{
server authservice:5001;
}
upstream portal
{
server portalservice:5010;
}
server
{
listen 44343 http2 ssl;
server_name id-local.com;
location /
{
proxy_pass http://auth;
proxy_redirect off;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host:44343;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
}
}
server
{
listen 44343 http2 ssl;
server_name app-local.com;
location /
{
proxy_pass http://portal;
proxy_redirect off;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host:44343;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name:44343;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}
Issue 2 I tried to use Nginx as API gateway for API Service. And encounter another problem.
Error log
"GET /api/v1 HTTP/2.0" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
[error] 34#34: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.112.1, server: app-local..com, request: "GET /api/v1 HTTP/2.0", upstream: "http://123.456.112.6:5020/api/v1", host: "app-local.com:5200"
Part of Nginx Config file
upstream api
{
server apiservice:5020;
}
server
{
listen 44343 http2 ssl;
server_name app-local.com;
#include /etc/nginx/default.d/*.conf;
location /api/v1
{
proxy_pass http://api/api/v1;
proxy_redirect off;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host:44343;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name:44343;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}
From the error, it seems like Nginx is unable to access the API Service‘s container. What is puzzling; when using Postman, I am able to get the data via http://localhost:5020/api/v1.
Part of docker-compose.yml
version: '3.9'
services:
nginxservice:
build:
context: .
dockerfile:
depends_on:
- authservice
ports:
- "5200:44343"
apiservice:
image: ${DOCKER_REGISTRY-}
build:
context: .
dockerfile:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://+:80
ports:
- 5020:80
portalservice:
image: ${DOCKER_REGISTRY-}
build:
context: .
dockerfile:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://*:5010
ports:
- 5010:5010
authservice:
image: ${DOCKER_REGISTRY-}
build:
context: .
dockerfile:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://*:5001
restart: always
ports:
- 5001:5001
Appreciated everyone to help
*EDIT: 14 Jul 2021: To add the more details especially on the setup of the services and the server name.
2
Answers
I've resolve the issue. The problem is drum rolls due to the mapping of the expose port number in Nginx Service.
It seems that for any docker services (correct me if I'm wrong), the mapping of the expose port number and the internal container's port number must be the same.
Hence instead of
"5200:44343", it should be"44343:44343"A couple of things that caught my eye.
http://app-local..com:44343seems to be contradicting the endpoint in your image which has theid-localaccessed via https.id-local.comto be listening on port44343. Are you configuring it correctly in your oidc client application? (can’t really see from the screenshot as some parts of the URL is hidden)