Question 395537 in Docker Question posted in
The official documentation can be found here.

Docker – how to pull model automatically with container creation?

When trying to access the ollama container from another (node) service in my docker compose setup, I get the following error:

ResponseError: model ‘llama3’ not found, try pulling it first

I want the setup for the containers to be automatic and don’t want to manually connect to the containers and manually pull the models.
Is there a way to load the model of my choice automatically when I create the ollama docker container?

Here is my relevant part of the docker-compose.yml

ollama:
        image: ollama/ollama:latest
        ports:
            - 11434:11434
        volumes:
            - ./ollama/ollama:/root/.ollama
        container_name: ollama
        pull_policy: always
        tty: true
        restart: always
Tags:

2

Answers


  1. 0

    Use a custom entrypoint script to download the model when a container is launched. The model will be persisted in the volume mount, so this will go quickly with subsequent starts.

    version: '3.7'

services:
  ollama:
    image: ollama/ollama:latest
    ports:
        - 11434:11434
    volumes:
        - ./ollama/ollama:/root/.ollama
        - ./entrypoint.sh:/entrypoint.sh
    container_name: ollama
    pull_policy: always
    tty: true
    restart: always
    entrypoint: ["/usr/bin/bash", "/entrypoint.sh"]

    Key changes:

    1. Added an entrypoint.
    2. Added a volume mount for entrypoint.sh.

    This is the content of entrypoint.sh:

    #!/bin/bash

# Start Ollama in the background.
/bin/ollama serve &
# Record Process ID.
pid=$!

# Pause for Ollama to start.
sleep 5

echo "🔴 Retrieve LLAMA3 model..."
ollama pull llama3
echo "🟢 Done!"

# Wait for Ollama process to finish.
wait $pid

    enter image description here

    Login or Signup to reply.
  2. 0

    This is what worked for me without needing an additional mount for the entry script.

    services:
  ollama:
    container_name: ollama
    pull_policy: always
    build:
      context: .
      dockerfile: ollama.dockerfile
    volumes:
      - type: bind
        source: [your local path here]
        target: /root/.ollama
    ports:
      - "11434:11434"
    restart: always

    FROM ollama/ollama

# Copy the script to the docker image
COPY ./wait_for_ollama.sh /wait_for_ollama.sh

# Ensure the script is executable
RUN chmod +x /wait_for_ollama.sh

EXPOSE 11434
ENTRYPOINT ["/bin/sh", "/wait_for_ollama.sh"]

    Then I used the script provided by @datawookie as wait_for_ollama.sh in the same directory as the dockerfile.

    #!/bin/bash

# Start Ollama in the background.
ollama serve &
# Record Process ID.
pid=$!

# Pause for Ollama to start.
sleep 5

echo "🔴 Retrieving model..."
ollama pull phi3:medium
echo "🟢 Done!"

# Wait for Ollama process to finish.
wait $pid
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search