Question 336249 in Docker Question posted in
The official documentation can be found here.

Docker – How to compose Ollama server with Nextjs?

I have been attempting to create a composition involving the Ollama server with LLMs such as Mistral and Llama2, along with a Next.js server to interact with it. I am building this project to learn Docker, and I am currently facing an issue.

So I have created a docker file for Nextjs like this :

FROM node:18-alpine

WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install

COPY . .
RUN npm run build
CMD ["npm", "start"]

Along with this I seek to run a llama server with cached dependencies. For this I tried to create a docker-compose file :

version: '3.8'
services:
  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama:/root/.ollama
    command: ollama pull llama2

  ollama-ui:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "3000:3000"

This way I am trying to run a command called ollama pull llama2, intending for llama2 to be pulled before it starts Nextjs server. But there is an error :

ollama-1 | Error: unknown command "ollama" for "ollama".

Goal of this project is a composed container running ollama server with some LLMs running along with my Nextjs server and I want the process of pulling LLMs to be cached. Is this a proper approach? If not what should it be?

I am sure this is some silly problem but will be glad if someone schools me here.

Thanks!

Tags:

2

Answers


  1. 0

    I have faced the same problem and found that despite being in /bin, compose can’t find the ollama command while running the image. One hack I’ve found around this is to run a script after docker compose is running to pull llama2.

    #!/usr/bin/bash

curl -X POST 
-H "Content-Type: application/json" 
-d '{"name":"llama2"}' 
 http://localhost:11434/api/pull
    Login or Signup to reply.
  2. 0

    In case anyone is still looking for a better solution, the issue is that the docker image’s entrypoint is already the ollama command, so you can just directly do pull llama2 without the ollama bit.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search