I have a docker project which uses docker-compose to stand up an API container and a database container. I also have it setup to create volumes for uploads
and logs
. When I build a new image, and then stand it up using docker-compose up
, any data that was previously in the logs
or uploads
ceases to exist, but the data in our db-data
volume persists. What do I need to change in my process and/or config to preserve the data in those other two volumes?
Here’s how we stand up the docker:
docker-compose down
docker build -t api:latest .
docker save -o api.tar api:latest postgres:14
docker load -i api.tar
docker-compose up -d
Here’s the Dockerfile:
# syntax=docker/dockerfile:1
FROM cupy/cupy AS deps
COPY nodesource_setup.sh .
RUN bash nodesource_setup.sh
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install libssl-dev ca-certificates cmake nodejs libgl-dev libglib2.0-0 -y
RUN pip3 install opencv-python python-dotenv
WORKDIR /app
COPY ./python ./ml_services/
COPY ./api/ .
RUN npm config set unsafe-perm true
RUN npm ci
# Rebuild the source only when needed
FROM deps AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=deps /app .
ENV NEXT_TELEMETRY_DISABLED=1
RUN npm run build
FROM deps AS runner
WORKDIR /app
ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED=1
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=builder /app/ml_services ./ml_services
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/.env.production.local ./.env
COPY --from=builder /app/prisma ./prisma
COPY --from=builder /app/.next ./.next
RUN mkdir -p /logs
RUN mkdir -p /uploads
RUN chown -R nextjs ./ /logs /uploads
# Cleanup
RUN apt-get clean && rm -rf /var/lib/apt
USER nextjs
EXPOSE 3000
ENV PORT 3000
CMD ["npm", "run", "deploy"]
And the docker-compose.yml
version: "3.7"
services:
database:
image: postgres:14
restart: always
env_file: .env.production.local
container_name: postgres
healthcheck:
test: "pg_isready --username=autocal && psql --username=autocal --list"
timeout: 10s
retries: 20
volumes:
- db-data:/var/lib/postgresql/data
api:
image: api:latest
ports:
- 3000:3000
depends_on: [database]
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
- uploads:/uploads
- logs:/logs
volumes:
db-data:
uploads:
logs:
2
Answers
Absolutely ended up being a code issue in the API layer. Sigh. Thanks all.
Make sure you writing
logs
anduploads
in/logs
and/uploads
in your API code.It might be the case it is being written it in
~/logs
and~/uploads
(in the home directory of execution user).Try to use absolute path not the relative path in your code.
Please go through node js docs around the path module https://nodejs.org/api/path.html