Question 97243 in Nginx Question posted in
The official documentation can be found here.

How to set nginx proxy path to use static contents of a specific s3 bucket folder?

I’m working on an nginx reverse proxy container image to proxy frontend files from s3, and Im trying to access these files from a specific folder location, instead of just the base path of the s3 bucket. As of yet I can only serve up the index.html which I’m using a rewrite for, but I’m getting a 403 on the js and css files.

I’ve tried including mime.types

include       mime.types;

I’ve tried adding an s3 folder bucket param

proxy_pass http://YOURBUCKET.s3-website.eu-central-1.amazonaws.com/$1;

And then various regex options

Here is my nginx conf file

server {
    listen 80;
    listen  443 ssl;
    ssl_certificate     /etc/ssl/nginx-server.crt;
    ssl_certificate_key   /etc/ssl/nginx-server.key;

    server_name timemachine.com;
    sendfile on;
    default_type application/octet-stream;
    resolver        8.8.8.8;
    server_tokens   off;

    location ~ ^/app1/(.*) {
        set $s3_bucket_endpoint  "timemachineapp.s3-us-east-1.amazonaws.com";
        proxy_http_version     1.1;
        proxy_buffering        off;
        proxy_ignore_headers   "Set-Cookie";
        proxy_hide_header      x-amz-id-2;
        proxy_hide_header      x-amz-request-id;
        proxy_hide_header      x-amz-meta-s3cmd-attrs;
        proxy_hide_header      Set-Cookie;
        proxy_set_header       Authorization "";
        proxy_intercept_errors on;
        rewrite ^/app1/?$ /dev/app1/index.html;    <-- I can only access index.html and the other js and css files throw a 403
        proxy_pass https://timemachineapp.s3-us-east-1.amazonaws.com;
        break;
    }
}

As you can see, I’m trying to make this so that the user goes to https://timemachine/app1 that this will go to the homepage and load all the css and js files. Again, what im getting is a 403 and sometimes a 404. Insight appreciated.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Alright so found a solution. Unless I'm missing something, this is easier than thought. For my use case, all I had to do was simply add multiple writes with those css files passed in (I'm sure there's a simpler way to just specify any .css file extension regardless of the naming of the file. Anyway, here is solution at the moment:

    server {
    listen 80;
    listen  443 ssl;
    ssl_certificate     /etc/ssl/nginx-server.crt;
    ssl_certificate_key   /etc/ssl/nginx-server.key;

    server_name timemachine.com;
    sendfile on;
    default_type application/octet-stream;
    resolver        8.8.8.8;
    server_tokens   off;

    location ~ ^/app1/(.*) {
        set $s3_bucket_endpoint  "timemachineapp.s3-us-east-1.amazonaws.com";
        proxy_http_version     1.1;
        proxy_buffering        off;
        proxy_ignore_headers   "Set-Cookie";
        proxy_hide_header      x-amz-id-2;
        proxy_hide_header      x-amz-request-id;
        proxy_hide_header      x-amz-meta-s3cmd-attrs;
        proxy_hide_header      Set-Cookie;
        proxy_set_header       Authorization "";
        proxy_intercept_errors on;
        rewrite ^/app1/?$ /dev/app1/index.html;   
        rewrite ^/app1/?$ /dev/app1/cssfile.css;  <- and keep adding, if needed
        proxy_pass https://timemachineapp.s3-us-east-1.amazonaws.com;
        break;
    }
}

  2. 0

    From the question it looks like

    • There’s a constant request-url prefix /app1/
    • There’s a constant proxied-url prefix /dev/app1/

    On that basis…

    First, enable the debug log

    There will already be an error_log directive in the nginx config, locate it and temporarily change to debug:

    error_log  /dev/stderr debug;

    This will allow you to see how these requests are being processed.

    Try naive-simple first

    Let’s use this config (other header directives omitted for brevity):

    location = /app1 { # redirect for consistency
    return 301 /app1/;
}

location = /app1/ { # explicitly handle the 'index' request
    proxy_pass https://example.com/dev/app1/index.html;
}

location /app1/ {
    proxy_pass https://example.com/dev/;
}

    And emit a request to it:

    $ ~ curl -I http://test-nginx/app1/some/path/some-file.txt
HTTP/1.1 403 Forbidden
...

    Note that S3 returns a 403 for requests that don’t exist, nginx is just proxying that response here.

    Let’s look in the logs to see what happened:

    2023/01/28 14:46:10 [debug] 15#0: *1 test location: "/"
2023/01/28 14:46:10 [debug] 15#0: *1 test location: "app1/"
2023/01/28 14:46:10 [debug] 15#0: *1 using configuration "/app1/"
...
"HEAD /dev/some/path/some-file.txt HTTP/1.0
Host: example.com
Connection: close
User-Agent: curl/7.79.1
Accept: */*
"

    So our request became https://example.com/dev/some/path/some-file.txt

    That’s because the way proxy_pass works is:

    If the proxy_pass directive is specified with a URI, then when a request is passed to the server, the part of a normalized request URI matching the location is replaced by a URI specified in the directive

    Meaning:

    Nginx receives:
/app1/some/path/some-file.txt
      ^ the normalized path starts here

Proxied-upstream receives:
/dev/some/path/some-file.txt
     ^ and was appended to proxy-pass URI

    I point this out as renaming/moving things on s3 may lead to a simpler nginx setup.

    Rewrite all paths, not specific requests

    Modifying the config above like so:

    location = /app1 { # redirect for consistency
    return 301 /app1/;
}

location = /app1/ { # explicitly handle the 'index' request
    proxy_pass https://example.com/dev/app1/index.html;
}

location /app1/ {
    rewrite ^/(.*) /dev/$1 break;             # prepend with /dev/
    # rewrite ^/app1/(.*) /dev/app1/$1 break; # OR this
    proxy_pass https://example.com/;          # no path here
}

    And trying that test-request again yields the following logs:

    "HEAD /dev/app1/some/path/some-file.txt HTTP/1.0
Host: example.com
Connection: close
User-Agent: curl/7.79.1
Accept: */*

"

    In this way the index request works, but also arbitrary paths – and there’s no need to modify this config to handle each individual url requested.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search