I have recently changed my OS to Debian 12. Before, when I wanted to install some python package, I used pip install <package>. But now, I get this error message:
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
What is the best way to handle python packages? Now I am installing them system-wide when it is possible (apt install -y python3-<package>) but sometimes the packages are not available in my repo. I have read something about venv but I do not understand it pretty well. If I follow the steps for installing packages using virtual environments, a directory called .venv is created, but should I create this directory on my HOME or on each python project folder? And what if I need to execute a python program as a root user? Will it take the virtual environment from the user HOME directory?
I want to learn how to handle python the best "healthy" way, as for now, I guess that the best option is to install the package system-wide, but when is not possible, how should I handle it? Using the --break-system-packages just does not seem correct.
2
Answers
If you are using some Python programs (not developing them), I recommend pipx for installing them. Pipx will create a virtualenv for your tool.
For installing pipx:
and after for its use:
upgrade of installed package is done like this:
What a great question!
As you suspect, that is not a good idea. Mainly because your OS may contain utilities that are written in Python, and if you mess up the system python environment, you could really mess up your computer. (I have done this!)
A venv (virtual environment) is just another installation of Python. If you look inside that .venv directory, you’ll find a complete installation…there’s a "bin" directory where a python executable will be installed, there’s a "lib" directory where you’ll find the packages installed in your virtualenv (it’s a few levels down, but it’s there). Go poke around…it’s a good learning experience.
Not a great idea, but if you must…
You can just specify the full path to the .venv version of python. So your root user would run something like
/home/0x141/myproject/.venv/bin/python /home/0x141/myproject/do_something.pyAnd it will probably pick up everything.
To be sure you pick up everything, you would want to "activate" your venv first. Every virtual environment manager has a slightly different way of doing it, but it would probably be something like
and so finally, my last piece of advice is to pick a virtual environment manager (virtualenv, virtualenvwrapper, pipenv, poetry) and play with it a little. Virtual environments are one of those things that are best learned by example, and the learning curve is short.
Good luck, and have fun!