Question 83335 in Docker Question posted in
The official documentation can be found here.

Installing python package from private gitlab repo in Dockerfile

I’m currently trying to install python packages from a private gitlab repo. Unfortunately, I get problems with the credentials. Is there any way to install this package without writing my credentials into the Dockerfile or adding my personal ssh key into it?

Dockerfile:

FROM python:3.9.12-buster AS production

RUN apt-get update && apt-get install -y git

COPY ./requirements.txt /app/requirements.txt

RUN pip install -r /app/requirements.txt

requirements.txt:

fastapi
uvicorn
cycler~=0.10.0
networkx
python-multipart
git+https://gitlab.private.net/group/private-repo.git@commit_hash#egg=foo

Error message:

#10 3.760   Cloning https://gitlab.private.net/group/private-repo.git (to revision commit_hash) to /tmp/pip-install-q9wtmf_q/foo_commit_hash     
#10 3.769   Running command git clone --filter=blob:none --quiet https://gitlab.private.net/group/private-repo.git /tmp/pip-install-q9wtmf_q/foo_commit_hash
#10 4.039   fatal: could not read Username for 'https://gitlab.private.net/group/private-repo.git': No such device or address
#10 4.060   error: subprocess-exited-with-error
Tags:

2

Answers


  1. 0

    Generally speaking, you can use multi-stage docker builds to make sure your credentials don’t stay in the image.

    In your case, you might do something like this:

    FROM python:3.9.12-buster as download
RUN apt-get update && apt-get install -y git
RUN pip install --upgrade pip wheel
ARG GIT_USERNAME
ARG GIT_PASSWORD

WORKDIR /build
COPY requirements.txt .
# add password to requirements file
RUN sed -i -E "s|gitlab.private.net|$GIT_USERNAME:[email protected]|" requirements.txt

# download dependencies and build wheels to /build/dist
RUN python -m pip wheel -w /build/dist -r requirements.txt

FROM python:3.9.12-buster as production
WORKDIR /app
COPY --from=download /build/dist /wheelhouse
# install dependencies from the wheels created in previous build stage
RUN pip install --no-index /wheelhouse/*.whl

COPY . .
# ... the rest of your dockerfile

    In GitLab CI, you might use the build command like this:

    script:
  # ...
  - docker build --build-arg GIT_USERNAME=gitlab-ci-token --build-arg GIT_PASSWORD=$CI_JOB_TOKEN -t $CI_REGISTRY_IMAGE .

    Then your image will be built and the final image won’t contain your credentials. It will also be smaller since you don’t have to install git 🙂

    As a side note, you can simplify this somewhat by using the GitLab PyPI package registry.

    Login or Signup to reply.
  2. 0

    So I also had to install my dependencies from private package repository for my python project.
    This was the Dockerfile I used for building my project.

    ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
RUN apt-get update &&
    apt-get install -y binutils libproj-dev gettext gcc libpq-dev python3-dev build-essential python3-pip python3-setuptools python3-wheel python3-cffi libcairo2 libpango-1.0-0 libpangocairo-1.0-0 libgdk-pixbuf2.0-0 libffi-dev shared-mime-info

RUN pip config set global.extra-index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.com/simple/  
# you need to configure pip to pull packages from remote private repository.
# for gitlab you require personal access token to access them with read permissions


COPY . /code/

RUN --mount=type=cache,target=/root/.cache pip install -r requirements.txt

RUN --mount=type=cache,target=/root/.cache pip install -r /code/webapi/requirements.txt

WORKDIR /code/webapi

ENTRYPOINT /code/webapi/entrypoint.sh
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search