I have a simple .gitlab-ci.yml
script that builds my python project.
image: python:3.9.6-slim-buster
variables:
PIP_DEFAULT_TIMEOUT: 300
before_script:
- pip install poetry==1.1.7
- poetry config virtualenvs.create false
- poetry install
When I run the CI pipeline, I periodically get such errors and the job is interrupted with a failure.
First type of error:
...
• Installing toml (0.10.2)
• Installing uvloop (0.16.0)
• Installing watchgod (0.8.2)
• Installing websockets (10.3)
ConnectionError
HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/flake8-eradicate/1.2.1/json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa5c5625dc0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
at /usr/local/lib/python3.9/site-packages/requests/adapters.py:565 in send
561│ if isinstance(e.reason, _SSLError):
562│ # This branch is for urllib3 v1.22 and later.
563│ raise SSLError(e, request=request)
564│
→ 565│ raise ConnectionError(e, request=request)
566│
567│ except ClosedPoolError as e:
568│ raise ConnectionError(e, request=request)
569│
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
Second type of error:
...
• Installing gitpython (3.1.27)
OSError
Could not find a suitable TLS CA certificate bundle, invalid path: /usr/local/lib/python3.9/site-packages/certifi/cacert.pem
at /usr/local/lib/python3.9/site-packages/requests/adapters.py:263 in cert_verify
259│ if not cert_loc:
260│ cert_loc = extract_zipped_paths(DEFAULT_CA_BUNDLE_PATH)
261│
262│ if not cert_loc or not os.path.exists(cert_loc):
→ 263│ raise OSError(
264│ f"Could not find a suitable TLS CA certificate bundle, "
265│ f"invalid path: {cert_loc}"
266│ )
267│
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1
What is most interesting is that these errors are triggered on completely different libraries at different times. I have to do a retry many times so that it installs everything stably. What could be the problem and how to solve it?
For info: I use dockerized gitlab runner with docker executor on CentOS 7
2
Answers
I think this has to do more with poetry.
Could be more likely due to parallel installations.
1.2
but since1.2
isn’t release you can use1.2.0b1
and checkCheck this out for more details https://github.com/python-poetry/poetry/issues/3336.
Regarding the
This is probably caused because poetry install is acting in the very environment where poetry is installed, causing it to uninstall its own dependencies because they aren’t listed in the lockfile. This is explicitly warned in CI recommendations in the installation guide:
https://python-poetry.org/docs/#ci-recommendations
You would notice because the logs from the install instruction start not creating a virtualenv and follow removing packages that are about to break the installation:
Solution:
and be careful using poetry inside tox (tox creates venv, tox installs poetry, poetry removes certifi)
Regarding the
I think this deserves a totally different Q&A, but this might simply be some PyPI downtime you unluckily observed.