docker build exactly same requirements.txt & Dockerfile, 4 weeks ago success, but currently failed

Steps to reproduce the failure.

Docker version 24.0.2

Python version in container 3.10.9 see base image

1. requirements.txt

pycurl==7.45.1
delta==0.4.2
fiftyone==0.21.0
filetype==1.2.0
Flask==2.2.2
kfp==1.8.19
kfp_pipeline_spec==0.1.16
numpy==1.23.5
pandas==1.5.2
Pillow==9.4.0
pyspark==3.3.2
requests==2.28.1
shortuuid==1.0.11
ultralytics==8.0.109
cvat_sdk==2.4.4
onnxruntime==1.14.1

PyJWT==2.7.0
bcrypt==4.0.1
email-validator==2.0.0.post2
decorator==5.1.1
Flask-Admin==1.5.8
Flask-CAS==1.0.2
Flask-Cors==3.0.10
Flask-JWT-Extended==4.4.4
Flask-Login==0.6.2
flask-mongoengine==1.0.0
Flask-Principal==0.4.0
Flask-PyMongo==2.3.0
Flask-RESTful==0.3.9
Flask-SQLAlchemy==3.0.3
Flask-WTF==1.1.1
SQLAlchemy==1.4.48
SQLAlchemy-Utils==0.41.1
Jinja2==3.0.3

2. Dockerfile

FROM pytorch/pytorch:2.0.0-cuda11.7-cudnn8-runtime

RUN apt-get update && apt-get install -y curl libcurl4-openssl-dev libssl-dev git gcc ffmpeg libsm6 libxext6 libgl1-mesa-glx wget -y && rm -rf /var/lib/apt/lists/*

RUN mkdir /app
WORKDIR /app

COPY . /app

RUN pip install pip --upgrade && pip install -r requirements.txt

3. command
   docker build -t env_test .

4. Error message

#0 33.30 Collecting PyYAML (from fiftyone==0.21.0->-r requirements.txt (line 3))
#0 33.31   Downloading PyYAML-5.4.1.tar.gz (175 kB)
#0 33.33      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 kB 15.4 MB/s eta 0:00:00
#0 33.49   Installing build dependencies: started
#0 36.84   Installing build dependencies: finished with status 'done'
#0 36.84   Getting requirements to build wheel: started
#0 37.05   Getting requirements to build wheel: finished with status 'error'
#0 37.06   error: subprocess-exited-with-error
#0 37.06
#0 37.06   × Getting requirements to build wheel did not run successfully.
#0 37.06   │ exit code: 1
#0 37.06   ╰─> [62 lines of output]
#0 37.06       /tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
#0 37.06       !!
#0 37.06
#0 37.06               ********************************************************************************
#0 37.06               The license_file parameter is deprecated, use license_files instead.
#0 37.06
#0 37.06               By 2023-Oct-30, you need to update your project and remove deprecated calls
#0 37.06               or your builds will no longer be supported.
#0 37.06
#0 37.06               See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
#0 37.06               ********************************************************************************
#0 37.06
#0 37.06       !!
#0 37.06         parsed = self.parsers.get(option_name, lambda x: x)(value)
#0 37.06       running egg_info
#0 37.06       writing lib3/PyYAML.egg-info/PKG-INFO
#0 37.06       writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
#0 37.06       writing top-level names to lib3/PyYAML.egg-info/top_level.txt
#0 37.06       Traceback (most recent call last):
#0 37.06         File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#0 37.06           main()
#0 37.06         File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#0 37.06           json_out['return_val'] = hook(**hook_input['kwargs'])
#0 37.06         File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
#0 37.06           return hook(config_settings)
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
#0 37.06           return self._get_build_requires(config_settings, requirements=['wheel'])
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
#0 37.06           self.run_setup()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
#0 37.06           exec(code, locals())
#0 37.06         File "<string>", line 271, in <module>
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
#0 37.06           return distutils.core.setup(**attrs)
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
#0 37.06           return run_commands(dist)
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
#0 37.06           dist.run_commands()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
#0 37.06           self.run_command(cmd)
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
#0 37.06           super().run_command(command)
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#0 37.06           cmd_obj.run()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 314, in run
#0 37.06           self.find_sources()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
#0 37.06           mm.run()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 551, in run
#0 37.06           self.add_defaults()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
#0 37.06           sdist.add_defaults(self)
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
#0 37.06           super().add_defaults()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
#0 37.06           self._add_defaults_ext()
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
#0 37.06           self.filelist.extend(build_ext.get_source_files())
#0 37.06         File "<string>", line 201, in get_source_files
#0 37.06         File "/tmp/pip-build-env-xd74p4sc/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
#0 37.06           raise AttributeError(attr)
#0 37.06       AttributeError: cython_sources
#0 37.06       [end of output]
#0 37.06
#0 37.06   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 37.06 error: subprocess-exited-with-error
#0 37.06
#0 37.06 × Getting requirements to build wheel did not run successfully.
#0 37.06 │ exit code: 1
#0 37.06 ╰─> See above for output.
#0 37.06
#0 37.06 note: This error originates from a subprocess, and is likely not a problem with pip.

I try to specified earch dependecies with exact version number, but there is still errors

It was very weird that 4 weeks ago I successfully build this environment into a docker image with exactly the same codes.

====== After I upgraded PyYAML to 6.0.1, still failes ======

1. add line PyYAML==6.0.1 in requirements.txt
2. run docker build again
3. Error Message

#0 103.8 Collecting absl-py<2,>=0.9 (from kfp==1.8.19->-r requirements.txt (line 7))
#0 103.8   Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
#0 103.8      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 3.1 MB/s eta 0:00:00
#0 103.8 INFO: pip is looking at multiple versions of kfp to determine which version is compatible with other requirements. This could take a while.
#0 103.8 ERROR: Cannot install -r requirements.txt (line 4), -r requirements.txt (line 7) and PyYAML==6.0.1 because these package versions have conflicting dependencies.
#0 103.8
#0 103.8 The conflict is caused by:
#0 103.8     The user requested PyYAML==6.0.1
#0 103.8     fiftyone 0.21.0 depends on PyYAML
#0 103.8     kfp 1.8.19 depends on PyYAML<6 and >=5.3
#0 103.8
#0 103.8 To fix this you could try to:
#0 103.8 1. loosen the range of package versions you've specified
#0 103.8 2. remove package versions to allow pip attempt to solve the dependency conflict

4. Let me try using PyYAML==5.3.1 instead
   Success!!!!!!

Because kfp 1.8.19 depends on PyYAML<6 and >=5.3 will be an constraint in another place