skip to Main Content

Issue:
pytest stopped generating coverage

Description:
I upgraded python version from 3.6 to 3.11 (Ubuntu 18.04). I followed this https://www.debugpoint.com/install-python-3-11-ubuntu/ in order to do that.
Had issues with pip, followed this advice: https://stackoverflow.com/a/72685573/15923186

I have a project with following structure:

test.sh
|-sorting
  |- env (the virtual environment for sorting)
  |- requirements.txt
  |- pyproject.toml
  |- __init__.py
  |- merge_sort.py
  |- tests
     |-__init__.py
     |-test_merge_sort.py
  
|-something_else (etc.)

requirements.txt are following:

attrs==22.1.0
black==22.8.0
click==8.0.4
coverage==6.2
flake8==5.0.4
importlib-metadata==4.2.0
iniconfig==1.1.1
mccabe==0.7.0
mypy-extensions==0.4.3
packaging==21.3
pathspec==0.9.0
platformdirs==2.4.0
pluggy==1.0.0
py==1.11.0
pycodestyle==2.9.1
pyflakes==2.5.0
pyparsing==3.0.9
pytest==7.0.1
pytest-cov==4.0.0
pytest-cover==3.0.0
tomli==1.2.3
typing-extensions==4.1.1
zipp==3.6.0

Pyproject is following:

[tool.pytest.ini_options]
pythonpath = [
  "."
]

The test script is following:

#!/bin/bash

source "$1"/env/bin/activate

cd "$1" || { echo "Project not found!"; exit 1;}

pytest -vv --cov="$1" --cov-report xml --cov-report term

And is invoked like this:
./test.sh sorting

I’m not "reusing" old virtual environments, I double checked. Before running tests I prepared a new one like this (the same as previously when using python 3.6):

python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

Python 3.11 is the default one for python3 command (output below):

> Python 3.11.0 (main, Oct 24 2022, 19:56:01) [GCC 7.5.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> 

Result:
Nothing displayed, no coverage report generated

Expected result:
Coverage report generated and displayed in the terminal

I also tried to run with:

python3 -m pytest -vv --cov="$1" --cov-report xml --cov-report term

No difference.

If I remove

 --cov="$1" --cov-report xml --cov-report term

from the test.sh script I get following output (which is good and expected)

test session starts ================================================================
platform linux -- Python 3.11.0, pytest-7.0.1, pluggy-1.0.0 -- /path/to/the/env/which/is/irrelevant/sorting/env/bin/python3
cachedir: .pytest_cache
rootdir: /home/mysusrname/folder_name/subfolder_name/sorting, configfile: pyproject.toml
plugins: cov-4.0.0
collected 3 items                                                                                                                                  

tests/test_merge_sort.py::test_merge_sort[numbers0] PASSED                                                                                   [ 33%]
tests/test_merge_sort.py::test_merge_sort[numbers1] PASSED                                                                                   [ 66%]
tests/test_merge_sort.py::test_merge_sort[numbers2] PASSED                                                                                   [100%]

3 passed in 0.01s =================================================================

2

Answers


  1. Chosen as BEST ANSWER

    So I couldn't agree with @Nathan and kept digging and actually my first comment is right and seems to be faster and imho cleaner solution.

    After upgrading python like I described and having a requirements.txt with fixed versions:

    • delete the old virtual environment

    • remove versions from requirements.txt (example below):

        attrs
        black
        click
        coverage
        flake8
        importlib-metadata
        iniconfig
        mccabe
        mypy-extensions
        packaging
        pathspec
        platformdirs
        pluggy
        py
        pycodestyle
        pyflakes
        pyparsing
        pytest
        pytest-cov
        pytest-cover
        tomli
        typing_extensions
        zipp
      
    • after creating a "fresh" venv and sourcing it run pip install --upgrade pip to make sure the pip is up to date

    • install the packages (utilizing the requirements.txt with NO versions specified in the venv mentioned above) pip install -r requirements.txt

    • the newest versions supported for your python distro will be used in such case

    • run pip freeze > requirements.txt to overwrite the requirements file and again have stable version of it with versions of libraries fixed (example corresponding "clean" file below)

        attrs==22.1.0
        black==22.12.0
        click==8.1.3
        coverage==6.5.0
        flake8==6.0.0
        importlib-metadata==5.1.0
        iniconfig==1.1.1
        mccabe==0.7.0
        mypy-extensions==0.4.3
        packaging==22.0
        pathspec==0.10.3
        platformdirs==2.6.0
        pluggy==1.0.0
        py==1.11.0
        pycodestyle==2.10.0
        pyflakes==3.0.1
        pyparsing==3.0.9
        pytest==7.2.0
        pytest-cov==4.0.0
        pytest-cover==3.0.0
        tomli==2.0.1
        typing_extensions==4.4.0
        zipp==3.11.0
      

    Side notes: I don't think having multiple versions of python is convenient to use, nor it is required in most cases actually. Probably one version as a "default"/to go and maybe a new release / release candidate as a second one is enough. Cannot imagine a situation where I need 5 or 6 versions...The venvs manage versions of python libraries and e.g. docker containers (or other virtualization method) can abstract anything else (not necessarily python related). Not to be misunderstood: Is @Nathan 's solution bad? Nope, it's good, but there are some points I disagree with, that's all. Also his comments helped me out to come up with a solution of my own, so upvote from me :)


  2. According to your requirement.txt, you are using pytest 7.0.1 which came out before python 3.11 (https://docs.pytest.org/en/stable/changelog.html#pytest-7-0-1-2022-02-11), which makes it unlikely to support it as it didn’t exist at that time. (N-B: sometimes it works, but it is not officially supported by the developers of pytest)

    For your case, as you need to upgrade several Python version at the time, I would use the following process to upgrade Python version and dependencies.

    • In Python 3.6, upgrade all dependencies to the latest version that supports Python 3.6
    • Test
    • Upgrade to Python 3.7
    • Test
    • In Python 3.7, upgrade all dependencies to the latest version that supports Python 3.7
    • Test
    • Keep going

    It is of course possible to skip some python version to speed up the process, but you might run into the same issues as you encountered.

    Usually, there are between 3 and 4 supported version of Python out at the same time (https://endoflife.date/python), which means that most library will support those versions (not always the case, for example numpy support less versions: https://numpy.org/neps/nep-0029-deprecation_policy.html), so it is usually possible to skip 1 version or 2. 3 is likely to not work.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search