I’m using Nginx OSS nginx/1.20.1 version as a reverse proxy and load balancer. Behind the load balancer, there are two servers available where the jfrog artifactory web app service has been running.
I would like to configure the Nginx in such a way that whenever the web app service gets down, the Nginx should stop sending the request to the particular server.
For this purpose, I have added the passive health check parameters(fail_timeout=900s) in the conf file
upstream artifactory {
server 172.1.1.1:8082 fail_timeout=900s;
server 172.1.1.2:8082 fail_timeout=900s;
}
upstream artifactory-direct {
server 172.1.1.1:8081 fail_timeout=900s;
server 172.1.1.2:8081 fail_timeout=900s;
}
server {
listen 80 default_server;
server_name xxx.xxx.xxxx.net xxx.xxx.xxx.xxx.net;
return 301 https://xxx.xxx.xxx.xxx.net$request_uri;
}
Then I have stopped the web service in one of the upstream servers and checked the Nginx error log, it is redirecting the request to both the upstream servers. Ideally, it should mark one of the servers as "unavailable" but it is doing like that. Could someone help with this, please?
Attached the nginx config files
2
Answers
This is indeed an expected behavior with Nginx open-source version. You can find more details about this on this page
As an alternate: you can automate the process of validating the backend (Artifactory) availability with a simple health check REST API and update the Nginx configuration if there is an issue. Keep running the script in a crontab
Here’s how Nginx works with the directive
fail_timeout
, assumefail_timeout
is set toTd
:U0
) fails to process and respond with number of requests (specified bymax_fails
) inTd
seconds, Nginx will markU0
as unavailable at the timeT0
.U0
forTd
seconds, until the timeout happenes (at the timeT0 + Td
), then Nginx retries connecting and sending a request toU0
:U0
is recovered and able to handle request, then mark it as activeU0
until next timeout event.To see whether Nginx marks any upstream server as unavailable, make sure the logging level is warn (or below that e.g. info, debug) so Nginx will dump appropriate detail to error log.