Question posted in
The official documentation can be found here.

Redis – Numpy installation failed on Mac M3

I am trying to install the dependencies for my Python project using the following command:

pip3 install -r requirements.txt

Requirement.txt

pyquaternion
importlib-metadata==4.13.0
Flask==2.1.1
celery==5.2.3
redis==3.2.1
Flask-Mail==0.9.1
psycopg2==2.7.7
psycopg2-binary==2.7.7
watchdog==0.9.0
SQLAlchemy==1.3.3
Flask-Migrate==2.4.0
alembic==1.0.10
python-json-logger==0.1.10
datetime==4.3
sentry-sdk==0.11.1
structlog==19.1.0
Flask-SQLAlchemy==2.4.0
requests==2.22.0
pytest==3.9.1
flake8==3.7.7
numpy==1.19.2
scipy==1.2.0
googledrivedownloader==0.3
tqdm==4.28.0
python-dotenv==0.10.2
json-logging==1.5.0-rc0
Flask-Cors==3.0.8
torch==1.11.0
scikit-image==0.14.2
pandas==1.3.5
scikit-learn==0.21.3
newrelic==6.4.4.161
Shapely==1.8.4
opencv-python==4.3.0.38
Werkzeug==2.0
lzf==0.1
backoff==1.10.0
Pillow==8.2.0
google-api-python-client
boto3
matplotlib
PyDrive
itsdangerous==2.0.1
torchvision>=0.5
protobuf<=3.20.1
open3d==0.16.0
python-bidi==v0.6.0 # To resolve compatibility issues with easyocr package
easyocr==1.4.1
google-cloud-documentai==2.18.0
redis==3.2.1
PyMuPDF==1.22.5
pygsheets==2.0.6
trimesh==4.2.0
pycocotools==2.0.7
pyannote.metrics==3.2.1
jiwer==3.0.1

When running the command, I encounter the following error while attempting to install numpy:

Collecting pyquaternion (from -r requirements.txt (line 1))
  Using cached pyquaternion-0.9.9-py3-none-any.whl.metadata (1.4 kB)
Collecting importlib-metadata==4.13.0 (from -r requirements.txt (line 2))
  Using cached importlib_metadata-4.13.0-py3-none-any.whl.metadata (4.9 kB)
Collecting Flask==2.1.1 (from -r requirements.txt (line 3))
  Using cached Flask-2.1.1-py3-none-any.whl.metadata (3.9 kB)
Collecting celery==5.2.3 (from -r requirements.txt (line 4))
  Using cached celery-5.2.3-py3-none-any.whl.metadata (20 kB)
Collecting redis==3.2.1 (from -r requirements.txt (line 5))
  Using cached redis-3.2.1-py2.py3-none-any.whl.metadata (33 kB)
Collecting Flask-Mail==0.9.1 (from -r requirements.txt (line 6))
  Using cached Flask-Mail-0.9.1.tar.gz (45 kB)
  Preparing metadata (setup.py) ... done
Collecting psycopg2==2.7.7 (from -r requirements.txt (line 7))
  Using cached psycopg2-2.7.7.tar.gz (427 kB)
  Preparing metadata (setup.py) ... done
Collecting psycopg2-binary==2.7.7 (from -r requirements.txt (line 8))
  Using cached psycopg2-binary-2.7.7.tar.gz (428 kB)
  Preparing metadata (setup.py) ... done
Collecting watchdog==0.9.0 (from -r requirements.txt (line 9))
  Using cached watchdog-0.9.0.tar.gz (85 kB)
  Preparing metadata (setup.py) ... done
Collecting SQLAlchemy==1.3.3 (from -r requirements.txt (line 10))
  Using cached SQLAlchemy-1.3.3.tar.gz (5.9 MB)
  Preparing metadata (setup.py) ... done
Collecting Flask-Migrate==2.4.0 (from -r requirements.txt (line 11))
  Using cached Flask_Migrate-2.4.0-py2.py3-none-any.whl.metadata (946 bytes)
Collecting alembic==1.0.10 (from -r requirements.txt (line 12))
  Using cached alembic-1.0.10.tar.gz (1.0 MB)
  Preparing metadata (setup.py) ... done
Collecting python-json-logger==0.1.10 (from -r requirements.txt (line 13))
  Using cached python_json_logger-0.1.10-py2.py3-none-any.whl.metadata (852 bytes)
Collecting datetime==4.3 (from -r requirements.txt (line 14))
  Using cached DateTime-4.3-py2.py3-none-any.whl.metadata (30 kB)
Collecting sentry-sdk==0.11.1 (from -r requirements.txt (line 15))
  Using cached sentry_sdk-0.11.1-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting structlog==19.1.0 (from -r requirements.txt (line 16))
  Using cached structlog-19.1.0-py2.py3-none-any.whl.metadata (9.4 kB)
Collecting Flask-SQLAlchemy==2.4.0 (from -r requirements.txt (line 17))
  Using cached Flask_SQLAlchemy-2.4.0-py2.py3-none-any.whl.metadata (3.1 kB)
Collecting requests==2.22.0 (from -r requirements.txt (line 18))
  Using cached requests-2.22.0-py2.py3-none-any.whl.metadata (5.5 kB)
Collecting pytest==3.9.1 (from -r requirements.txt (line 19))
  Using cached pytest-3.9.1-py2.py3-none-any.whl.metadata (5.5 kB)
Collecting flake8==3.7.7 (from -r requirements.txt (line 20))
  Using cached flake8-3.7.7-py2.py3-none-any.whl.metadata (3.6 kB)
Collecting numpy==1.19.2 (from -r requirements.txt (line 21))
  Using cached numpy-1.19.2.zip (7.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [54 lines of output]
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                  self.rng_state.ctr.v[i] = counter[i]
      
              self._reset_state_variables()
      
              self._bitgen.state = <void *>&self.rng_state
              self._bitgen.next_uint64 = &philox_uint64
                                         ^
      ------------------------------------------------------------
      
      _philox.pyx:195:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to the type of the value being assigned.
      Processing numpy/random/_bounded_integers.pxd.in
      Processing numpy/random/_philox.pyx
      Traceback (most recent call last):
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-install-8leg4th0/numpy_1464917656f54026a46d9416ca97ac20/tools/cythonize.py", line 235, in <module>
          main()
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-install-8leg4th0/numpy_1464917656f54026a46d9416ca97ac20/tools/cythonize.py", line 231, in main
          find_process_files(root_dir)
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-install-8leg4th0/numpy_1464917656f54026a46d9416ca97ac20/tools/cythonize.py", line 222, in find_process_files
          process(root_dir, fromfile, tofile, function, hash_db)
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-install-8leg4th0/numpy_1464917656f54026a46d9416ca97ac20/tools/cythonize.py", line 188, in process
          processor_function(fromfile, tofile)
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-install-8leg4th0/numpy_1464917656f54026a46d9416ca97ac20/tools/cythonize.py", line 78, in process_pyx
          [sys.executable, '-m', 'cython'] + flags + ["-o", tofile, fromfile])
        File "/Users/ravikumar/.pyenv/versions/3.7.12/lib/python3.7/subprocess.py", line 363, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/Users/ravikumar/workspace/alchemist/my_env/bin/python', '-m', 'cython', '-3', '--fast-fail', '-o', '_philox.c', '_philox.pyx']' returned non-zero exit status 1.
      Running from numpy source directory.
      setup.py:470: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
        run_build = parse_setuppy_commands()
      Cythonizing sources
      Traceback (most recent call last):
        File "/Users/ravikumar/workspace/alchemist/my_env/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/ravikumar/workspace/alchemist/my_env/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/ravikumar/workspace/alchemist/my_env/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-build-env-hsqnv_oc/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 157, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-build-env-hsqnv_oc/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 249, in run_setup
          self).run_setup(setup_script=setup_script)
        File "/private/var/folders/y4/_wxz69p51sn3p3tj133crn6r0000gn/T/pip-build-env-hsqnv_oc/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 142, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 499, in <module>
          setup_package()
        File "setup.py", line 479, in setup_package
          generate_cython()
        File "setup.py", line 274, in generate_cython
          raise RuntimeError("Running cythonize failed!")
      RuntimeError: Running cythonize failed!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

What I’ve Tried:

  1. Updating pip to the latest version.

  2. Manually installing problematic packages like numpy using --no-binary.

  3. Verified Python version compatibility with numpy.

Environment Details:

  • Python version: 3.7.12

  • pip version: 22.0.4

  • Operating System: macOS

Question: How can I resolve this error and successfully install the dependencies? Is there a specific workaround for handling pyproject.toml issues during the installation?

Tags:

2

Answers


  1. 0

    You have long list of specific version to work on so it is best to use a virtual env. I am fairly sure that you experince some version problems here. Not all package version play well with each other so virtual envs solve that.

    It looks like python 3.9 is suitable for your task. With similar versions i have encountered many problems after 3.11, but i am not sure for all of your requirements.

    So you can use conda or pyenv to generate your envoirment. it is best to do it in your project folder so open your folder and

    pyenv install 3.9.16
pyenv global 3.9 
python3.9 -m venv my_env 
source my_env/bin/activate

    The last line my_env/bin/activate is if you are on a unix system if you are on windows then use my_envScriptsactivate

    After you activate your env

    pip install --upgrade pip setuptools wheel

    it should set up you good for

    pip install -r requirements.txt

    if you are not too keen on using an env you can also download python3.9 and use commands like

    python3.9 -m pip install pacakge_name

    to specificlly install on that python verison and execute with python3.9 but this is not the safest solution and i only tried it on unix based system so i dont know if it works on windows.

    Login or Signup to reply.
  2. 0

    try the following steps (assuming you have python and pip is already installed)

    1. pip install uv
    2. uv init
    3. uv add -r requirements.txt (to download your requirements.txt) or uv add numpy (just for numpy)

    I tested out using buildx for arm64, and I don’t think the issue is due to your M3 Processor.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search