Question posted in
The official documentation can be found here.

Docker – Running REST API (Ollama) and Nginx reverse proxy

I want to run Ollama docker image. This exposes entire API with its endpoints.

I want to secure it by adding reverse proxy on top of it and i have chosen Nginx for the job.

So I need to add Nginx to my docker iamge with Ollama.

So i have constructed such Dockerfile:

FROM ollama/ollama:latest

FROM nginx:latest

COPY /.nginx/nginx.conf /etc/nginx/conf.d/default.conf

My Nginx configuration:

server {
    listen 80;
    
    location /api/tags {
        proxy_pass http://localhost:11434/api/tags;
        proxy_set_header Host localhost:11434;
    }
    
    # Default: Return 404 for all other endpoints
    location / {
        return 404;
    }
}

Here I allow just one endpoint to be proxied to Ollama.

In order to test it, I tried some endpoints and they returned correctly 404.

But when I have tried api/tags path, I got "Bad gateway" 502 error.

In docker logs I could see:

2024-12-01 22:44:41 2024/12/01 21:44:41 [warn] 29#29: *1 upstream server temporarily disabled while connecting to upstream, client: 172.17.0.1, server: , request: "GET /api/tags HTTP/1.1", upstream: "http://127.0.0.1:11434/api/tags", host: "localhost:8000"

2024-12-01 22:44:41 2024/12/01 21:44:41 [error] 29#29: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.17.0.1, server: , request: "GET /api/tags HTTP/1.1", upstream: "http://127.0.0.1:11434/api/tags", host: "localhost:8000"

2024-12-01 22:44:41 2024/12/01 21:44:41 [warn] 29#29: *1 upstream server temporarily disabled while connecting to upstream, client: 172.17.0.1, server: , request: "GET /api/tags HTTP/1.1", upstream: "http://127.0.0.1:11434/api/tags", host: "localhost:8000"

2024-12-01 22:44:41 172.17.0.1 – – [01/Dec/2024:21:44:41 +0000] "GET /api/tags HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36 OPR/114.0.0.0" "-"

It seems that Ollama server is not running, but I don’t know:

  1. How to check running processes in Docker, ps command is not found in docker.
  2. How to run Ollama server?
  3. How to define in Docker how start Ollama server automatically?
Tags:

2

Answers


  1. 0

    I think that the main problem is that the Dockerfile and the current configuration don’t allow both services (Ollama and Nginx) to be started simultaneously.

    Why ?

    (1) Incomplete Dockerfile

    The original only contains:

    FROM ollama/ollama:latest
FROM nginx:latest
COPY /.nginx/nginx.conf /etc/nginx/conf.d/default.conf

    This Dockerfile does not specify how to launch the services or how they will interact together.

    (2) The logs are clear about this

    In the provided logs, clear connection errors are visible:

    [error] 29#29: *1 connect() failed (111: Connection refused) while connecting to upstream

    This error indicates that Nginx cannot connect to Ollama, suggesting that the Ollama service is not started or not accessible.

    Login or Signup to reply.
  2. 0

    You can’t create a Docker image containing multiple programs the way you’re trying. The Dockerfile you have will result in an image that only contains Nginx and not Ollama. It’s also generally recommended to only have one process running in a container.

    I’d recommend running Ollama and Nginx in each their own containers and orchestrate them using Docker Compose.

    Create a docker-compose.yml file containing

    services:
  ollama:
    image: ollama/ollama:latest
  nginx:
    build: .
    ports:
      - 80:80

    Then change your nginx.conf file so it passes the request to the Ollama container by using the ollama service name instead of localhost like this:

    server {
    listen 80;
    
    location /api/tags {
        proxy_pass http://ollama:11434/api/tags;
        proxy_set_header Host localhost:11434;
    }
    
    # Default: Return 404 for all other endpoints
    location / {
        return 404;
    }
}

    Your Dockerfile will work as is, since the first line is effectively ignored. But to keep things tidy, you should change it to

    FROM nginx:latest

COPY /.nginx/nginx.conf /etc/nginx/conf.d/default.conf

    docker compose up -d will start both containers. When a request comes in to Nginx on http://localhost/api/tags it’ll pass it on to the Ollama container on /api/tags on port 11434.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search