skip to Main Content

I have a docker project which uses docker-compose to stand up an API container and a database container. I also have it setup to create volumes for uploads and logs. When I build a new image, and then stand it up using docker-compose up, any data that was previously in the logs or uploads ceases to exist, but the data in our db-data volume persists. What do I need to change in my process and/or config to preserve the data in those other two volumes?

Here’s how we stand up the docker:

docker-compose down
docker build -t api:latest .
docker save -o api.tar api:latest postgres:14
docker load -i api.tar
docker-compose up -d

Here’s the Dockerfile:

# syntax=docker/dockerfile:1
FROM cupy/cupy AS deps
COPY nodesource_setup.sh .
RUN bash nodesource_setup.sh
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install libssl-dev ca-certificates cmake nodejs libgl-dev libglib2.0-0 -y
RUN pip3 install opencv-python python-dotenv

WORKDIR /app

COPY ./python ./ml_services/
COPY ./api/ .
RUN npm config set unsafe-perm true
RUN npm ci

# Rebuild the source only when needed
FROM deps AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=deps /app .

ENV NEXT_TELEMETRY_DISABLED=1
RUN npm run build

FROM deps AS runner
WORKDIR /app

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED=1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/ml_services ./ml_services
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/.env.production.local ./.env
COPY --from=builder /app/prisma ./prisma
COPY --from=builder /app/.next ./.next

RUN mkdir -p /logs
RUN mkdir -p /uploads

RUN chown -R nextjs ./ /logs /uploads

# Cleanup
RUN apt-get clean && rm -rf /var/lib/apt

USER nextjs

EXPOSE 3000
ENV PORT 3000

CMD ["npm", "run", "deploy"]

And the docker-compose.yml

version: "3.7"

services:
  database:
    image: postgres:14
    restart: always
    env_file: .env.production.local
    container_name: postgres
    healthcheck:
      test: "pg_isready --username=autocal && psql --username=autocal --list"
      timeout: 10s
      retries: 20
    volumes:
      - db-data:/var/lib/postgresql/data

  api:
    image: api:latest
    ports:
      - 3000:3000
    depends_on: [database]
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]
    volumes:
      - uploads:/uploads
      - logs:/logs

volumes:
  db-data:
  uploads:
  logs:

2

Answers


  1. Chosen as BEST ANSWER

    Absolutely ended up being a code issue in the API layer. Sigh. Thanks all.


  2. Make sure you writing logs and uploads in /logs and /uploads in your API code.

    It might be the case it is being written it in ~/logs and ~/uploads (in the home directory of execution user).

    Try to use absolute path not the relative path in your code.

    Please go through node js docs around the path module https://nodejs.org/api/path.html

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search