psutil fails to pip install inside python3 virtualenv: fatal error: Python.h: No such file or directory - CentOS

mrlitsta
February 26, 2020
218 views
0 votes
2 Answers

I created a Python3 virtualenv in order to run Tower jobs in python3 now that python2 is EOL. I followed the instructions here: https://docs.ansible.com/ansible-tower/latest/html/upgrade-migration-guide/virtualenv.html but psutil failed to install with this error:

python36/root/usr/include/python3.6m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.6/psutil/_psutil_common.o
    psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

This is how I created my virtualenv:

sudo bash
mkdir /opt/my-envs/custom-venv 
virtualenv -p /opt/rh/rh-python36/root/usr/bin/python3 /opt/my-envs/custom-venv/ 
source /opt/my-envs/custom-venv/bin/activate 
pip install psutil ansible awscli boto boto3 botocore six urllib3 jmespath paramiko Jinja2

Others have suggested installing python3-devel but that’s already installed.

Environment:

CentOS 7.7
python3-devel-3.6.8
Python 3.6.8
pip3 9.0.3
Ansible Tower 3.5.0

More complete error trace:

# pip install psutil
Collecting psutil
  Downloading <CENSORED>/psutil-5.7.0.tar.gz (449kB)
    100% |████████████████████████████████| 450kB 52.5MB/s
Building wheels for collected packages: psutil
  Running setup.py bdist_wheel for psutil ... error

...

    gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -I/opt/rh/rh-python36/root/usr/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_LINUX=1 -I/opt/rh/rh-python36/root/usr/include/python3.6m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.6/psutil/_psutil_common.o
    psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/opt/my-envs/custom-venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-gv49hqf0/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('rn', 'n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-4oyv7j_1-record/install-record.txt --single-version-externally-managed --compile --install-headers /opt/my-envs/custom-venv/include/site/python3.6/psutil" failed with error code 1 in /tmp/pip-build-gv49hqf0/psutil/

Answers

Chosen as BEST ANSWER
- mrlitsta
- February 26, 2020 at 11:37 pm
- 0 votes
0
It turns out the virtualenv command is a python2 construct. The Ansible Tower docs are wrong in their recommendation: https://docs.ansible.com/ansible-tower/latest/html/upgrade-migration-guide/virtualenv.html
```
sudo virtualenv -p /opt/rh/rh-python36/root/usr/bin/python3 /opt/my-envs/custom-venv
source /opt/my-envs/custom-venv/bin/activate 
sudo /opt/my-envs/custom-venv/bin/pip install psutil
```
Use the Python3 venv module instead:
```
sudo python3 -m venv /opt/my-envs/custom-venv
source /opt/my-envs/custom-venv/bin/activate 
sudo /opt/my-envs/custom-venv/bin/pip install psutil
```