Question 380136 in Docker Question posted in
The official documentation can be found here.

Problem installing `apache-flink:1.19.0` using python docker 3.9 – 3.12

I am having trouble installing PyFlink/apache-flink 1.19.0 using python docker 3.9 – 3.12 following the official tutorial.

It seems that the error is about a path that return NoneType, but I am not sure how to fix it.

I did not post python:3.12 as the error is the same as others.

Can you please help me on this?

Thank you very much.

Python 3.9

docker run - it python:3.9 python - m pip install apache-flink

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-rdoi1tww/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
        File "/tmp/pip-build-env-rdoi1tww/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-rdoi1tww/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 267, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-rdoi1tww/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 169, in <module>
          include_dirs=get_java_include() + ['src/main/c/pemja/core/include'],
        File "setup.py", line 111, in get_java_include
          inc = os.path.join(get_java_home(), inc_name)
        File "/usr/local/lib/python3.9/posixpath.py", line 76, in join
          a = os.fspath(a)
      TypeError: expected str, bytes or os.PathLike object, not NoneType
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

Python 3.10

docker run - it python:3.10 python - m pip install apache-flink

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-ptcni4h1/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
        File "/tmp/pip-build-env-ptcni4h1/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-ptcni4h1/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 267, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-ptcni4h1/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 169, in <module>
          include_dirs=get_java_include() + ['src/main/c/pemja/core/include'],
        File "setup.py", line 111, in get_java_include
          inc = os.path.join(get_java_home(), inc_name)
        File "/usr/local/lib/python3.10/posixpath.py", line 76, in join
          a = os.fspath(a)
      TypeError: expected str, bytes or os.PathLike object, not NoneType
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Python 3.11

docker run - it python:3.11 python - m pip install apache-flink

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jpt4cjc2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jpt4cjc2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-jpt4cjc2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 268, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jpt4cjc2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 169, in <module>
          include_dirs=get_java_include() + ['src/main/c/pemja/core/include'],
                       ^^^^^^^^^^^^^^^^^^
        File "setup.py", line 111, in get_java_include
          inc = os.path.join(get_java_home(), inc_name)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen posixpath>", line 76, in join
      TypeError: expected str, bytes or os.PathLike object, not NoneType
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

UPDATE 15/4/2024

It works in Windows. But it does not work in Mac when using this command: Is it a permission problem?

docker run -it python:3.11 pip install apache-flink

My Mac Config:

  • M1 Max
  • Sonoma 14.4.1

Message below:

× Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-r8xb1epe/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-r8xb1epe/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-r8xb1epe/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 268, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-r8xb1epe/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 169, in <module>
          include_dirs=get_java_include() + ['src/main/c/pemja/core/include'],
                       ^^^^^^^^^^^^^^^^^^
        File "setup.py", line 111, in get_java_include
          inc = os.path.join(get_java_home(), inc_name)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen posixpath>", line 76, in join
      TypeError: expected str, bytes or os.PathLike object, not NoneType
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    For those who are having this problem like me, who are trying to install apache-flink in Macbook M1 via docker, I found the answer.

    The root cause: pemja requires JAVA_HOME to be set

    So all you need to do is to install JAVA jdk (not jre) first:

    • I am using openjdk 17. That is the only openJDK I found in Python:3.11 image.
      • To install, run apt install -y openjdk-17-jdk
    • Set JAVA_HOME variable by: export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64.
      • This is the location I found, your's may be different. Look it up when necessary.
    • Install pemja: pip install pemja.
    • If the above success, run pip install apache-flink

    Then you should be able to install apache-flink client successfully.

    Then last step is to commit this image and create your own docker image:

    • docker commit -m "fix apche-flink install issue" <your container name> <your_docker_image_name>:<your_docker_image_tag>

    Hope it helps someone.


  2. 0

    These commands should work:

    docker run -it python:3.9 pip install apache-flink
docker run -it python:3.10 pip install apache-flink
docker run -it python:3.11 pip install apache-flink

    The apache-flink package will be installed onto the running container in all cases. However, immediately after the package is installed the container will terminate, so it’s not terribly useful.

    What would make more sense would be to create a new image with the package installed.

    🗎 Dockerfile

    FROM python:3.11

RUN pip install apache-flink==1.19.0

    The build and run.

    docker build -t python-flink .
docker run -it python-flink

    In the running you container you will be able to import pyflink.

    enter image description here

    I did not post python:3.12 as the error is the same as others.

    Are you sure? I actually get a different error with Python 3.12 that looks like a version compatibility issue with the apache-flink package.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search