Question 217016 in Docker Question posted in
The official documentation can be found here.

Cannot resolve numpy and other packages version dependency using Docker

I am trying to setup a Docker execution of a python code but numpy keeps giving dependecies issues with other package versions.

Here is my Dockerfile (this one is using 2 stage execution but I have tried a single stage execution using both python 3.11 and python 3.11-slim):

# Stage 1
FROM python:3.11 as builder

#tried FROM python:3.11 and FROM python:3.11-slim as well

# working directory
WORKDIR /app

RUN apt-get update && apt-get install -y 
    build-essential 
    && rm -rf /var/lib/apt/lists/*


COPY requirements.txt .

#virtual environment
RUN python -m venv venv
ENV PATH="/app/venv/bin:$PATH"

RUN pip install --no-cache-dir --upgrade pip
RUN pip install --no-cache-dir -r requirements_new.txt
COPY . .

# Stage 2. Tried without this as well in a single stage execution
FROM python:3.11-slim

# Set the working directory inside the container
WORKDIR /app

COPY --from=builder /app .


# Run command
CMD [ "python", "./test.py"]

When I run this Dockerfile, I get following error:

ERROR: Cannot install -r requirements.txt because these package versions have conflicting dependencies.
The conflict is caused by:
 The user requested numpy==1.23.5
 argilla 1.13.2 depends on numpy<1.24.0
 contourpy 1.1.0 depends on numpy>=1.16
 langchain 0.0.200 depends on numpy<2 and >=1
 layoutparser 0.3.4 depends on numpy
 matplotlib 3.7.1 depends on numpy>=1.20
 numexpr 2.8.4 depends on numpy>=1.13.3
 onnxruntime 1.15.1 depends on numpy>=1.24.2

I have tried use single stage execution as well with base python 3.11 version as well as python 3.11-slim version, but both give the same dependency issue. Despite installing numpy 1.23.5 (which satisfies most of the package dependencies, it still gives error on those). Not sure how to get this issue resolved.

Tags:

2

Answers


  1. 0

    Python package management and dependencies can be nightmare. You should try poetry which is build to deal with dependenciess problems,read more basic usage poetry

    Try this tutorial
    Dependency Management With Python Poetry

    If you inherited a project that wasn’t created with Poetry, but now you want to use Poetry for your dependency management,you have that case also explained.

    Login or Signup to reply.
  2. 0

    The issue you’re facing is related to conflicting dependencies between different packages, specifically with numpy versions. Some of the packages in your requirements are expecting different versions of numpy, which creates a conflict.

    To resolve this issue, you can try the following approaches:

    1. Pin Numpy Version: Update your requirements_new.txt file (which you are using in the Dockerfile) to specify a compatible version of numpy that satisfies most of the package dependencies. For example, you can try pinning numpy to a version that is compatible with the majority of the packages in your requirements. You can also check if there are newer versions of the packages that support the latest numpy version and update the requirements accordingly.

    2. Upgrade Packages: Sometimes, upgrading certain packages to their latest versions can help resolve dependency conflicts. You can try upgrading some of the packages in your requirements_new.txt to see if that resolves the conflicts.

    3. Modify Package Requirements: If possible, try modifying the requirements of the packages causing conflicts. Some packages might allow a broader range of numpy versions. You can check the documentation or GitHub repository of those packages to see if there are any discussions on numpy version compatibility.

    4. Use Conda: Consider using Conda instead of pip for managing your package dependencies. Conda is a package manager that handles package dependencies and versions more effectively. It might help you avoid some of the conflicts.

    5. Isolate Packages: If none of the above solutions work, you can try isolating the conflicting packages in separate virtual environments within the Docker image. This might help prevent conflicts between them.

    Remember that Docker allows you to run commands interactively during the build process. You can use this to your advantage to debug the dependency issues step by step. For example, you can add a RUN command in the Dockerfile to install each package individually, which will help you identify which package is causing the conflict.

    In any case, it’s essential to carefully review the requirements of the packages you are installing and ensure that they are compatible with each other, especially when using a specific version of numpy.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search