Summary:
I want to make NGINX (not NGINX Plus) re-resolve the IP address from the DNS name by using a variable in proxy_pass
(as suggested in this official Nginx article in the section "Setting the Domain Name in a Variable"). But when I do that it won’t set/forward the correct Content-Type
header, but always use text/html
, for .css
, .js
etc. files.
My setup:
I have a frontend and a backend service running in separate Docker containers (to be deployed to OpenShift in production). There’s also a third container that runs nginx:latest
(v1.19.9 as of today) and acts as a reverse proxy, forwarding calls made to /my-app
to the frontend and /my-app/api
to the backend container. The NGINX reverse proxy has set them up as upstream servers using their DNS names.
All three containers run within the same custom Docker network, so resolving in itself works fine – but only, when NGINX is (re-)started.
The problem:
When the frontend or backend container gets restarted it may get a new IP address. Since NGINX caches IP addresses I’m getting a 502
when I make calls to them. I want NGINX to re-resolve the IP address more frequently, like NGINX Plus does. This is how the problem with the Content-Type
came up, when I tried to have NGINX re-resolve DNS names.
The configuration:
Here’s my NGINX configuration (simplified to the relevant stuff only):
index index.html index.htm;
upstream upstream_frontend {
server frontend:8080;
}
upstream upstream_backend {
server backend:8000;
}
server {
listen 8080;
root /usr/share/nginx/html;
try_files $uri$args $uri$args/ $uri $uri/ /index.html =404;
rewrite ^/my-app$ $scheme://$http_host/my-app/ permanent;
location /my-app {
resolver 127.0.0.11 ipv6=off valid=1s;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_http_version 1.1;
proxy_pass_request_headers on;
proxy_pass_header Content-Type;
# Attempt #1, NOT working: using a variable and an upstream server setup
set $frontend_var "http://upstream_frontend/";
proxy_pass http://$frontend_var;
# Attempt #2, NOT working: using a variable and directly using the container name
# set $frontend_var "http://frontend:8080/";
# proxy_pass $frontend_var;
# Attempt #3, working fine: using NO variable and an upstream server setup, , but no DNS re-resolving happening :-(
# proxy_pass http://upstream_frontend/;
# Attempt #4, working fine: using NO variable and directly using the container name, but no DNS re-resolving happening :-(
# proxy_pass http://frontend:8080/;
}
location /my-app/api {
resolver 127.0.0.11 ipv6=off valid=1s;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_redirect off;
# Attempt #1, NOT working: using a variable and an upstream server setup
set $backend_var "http://upstream_backend/api";
proxy_pass http://$backend_var;
# Attempt #2, NOT working: using a variable and directly using the container name
# set $backend_var "http://backend:8000/api";
# proxy_pass http://$backend_var;
# Attempt #3, working fine: using NO variable and an upstream server setup, , but no DNS re-resolving happening :-(
# proxy_pass http://upstream_backend/api;
# Attempt #4, working fine: using NO variable and directly using the container name, but no DNS re-resolving happening :-(
# proxy_pass http://backend:8000/api;
}
}
(Please note the location /my-app {...}
and the /
at the end of the proxy_pass
, as the frontend is served from /my-app
, whereas the frontend container itself, also served from NGINX, is operating directly on ...:8080/
, without the /my-app
context path. And we have the same setup for the backend container that is listening on :8000
, but when calls are forwarded to it we’re removing the /my-app
from them so they reach the container directly at ...:8000/api
.)
NGINX is running on port 9000
and when I open http://localhost:9000/my-app using the above Attempt #1 or Attempt #2 the .css
, .js
, image files etc. are all served with Content-Type: text/html
, which prevents the browser from properly rendering the page, and shows messages like this in the debugger pane:
The stylesheet http://localhost:9000/my-app/static/css/main.df1d2133.chunk.css was not loaded because its MIME type, ”text/html“ is not ”text/css"
At first I thought proxy_pass_header Content-Type;
would fix this, but that doesn’t help either.
And then I also learned from this NGINX bug ticket that "When using variables in proxy_pass, if URI is specified, it is passed to the server as is, replacing the original request URI.". This seems to be a possible cause for the Content-Type
issue I have here.
Also, as you can see in the configuration I have set the Docker DNS IP: resolver 127.0.0.11 ipv6=off valid=1s;
. I have also tried setting it outside the location
block, that also didn’t help.
Question:
So how do I have DNS re-resolving in NGINX (Open-Source) with the proper Content-Type
still set? I don’t have to use upstream servers, so I can get rid of them, should that be part of a possible fix.
P.S.:
I can’t, as suggested in other StackOverflow comments, add separate location
blocks to "retroactively" fix the Content-Type
depending on folder, file name etc., because this is an evolving project and I worry it would require me to keep adding such hotfixes to the NGINX configuration on a regular basis.
EDIT:
The configuration I posted is included by NGINX’s Docker container’s configuration file which is this one, in case you were wondering about a few "missing" settings:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# This line includes my configuration above
include /etc/nginx/conf.d/*.conf;
}
2
Answers
So here's my final working setup. Thanks again, @richard-smith!
Please note that...:
upstream
block as I'm deploying this to OpenShift. There's the concept of "Services" that fronts Pods (~= servers or containers), which is basically whatupstream
does in Nginx,location
blocks so the more specific one,/my-app/api
, gets hit first, as otherwise things will fail,$isargs$args
part as otherwise the URLs are not constructed properly and will fail, specifically for calls to my backend,.<PROJECT>.svc.cluster.local
on theproxy_pass
, as otherwise the resolver will not be able to resolve the service.This is the relevant setup, for the remaining (default) Nginx configuration see the configuration I posted in the EDIT of my initial question above.
This isn’t a problem with
Content-Type
. You are usingproxy_pass
incorrectly. As you have already noted:You currently have the following working configuration, but you want to replace it with a variable to force DNS resolution:
The simplest solution is to use
rewrite...break
to make the changes to the URI, and remove the trailing/
from the variable.For example:
Alternatively, capture the remainder of the original request using a regular expression
location
.For example: