Visual Studio Code Question posted in
View the official documentation.

Visual Studio Code – How to configure VS Code pytest when Python code is not in top level directory?

I started a project with what I would describe as a "standard" Python project structure, whereby my code is organized into a set of packages which all live in the top level directory. In addition to this, there is a tests directory which contains .py files with functions which define the test cases.

I would now like to segregate this from another new project which I will add to the same repository.

To do this, I propose to create two top level directories new_project and old_project under which everything will be organized. (Given the names, you could imagine this is part of some kind of migration.)

I don’t know how to make this work with pytest as it integrates with VS Code.

Here’s the proposed project structure, as a MWE:

new_project/
    simple_package/
        __init__.py
    tests/
        test_simple_package.py

old_project/
    # copy & paste of the above

The files can contain minimal code to demonstrate the point.

$ cat new_project/simple_package/__init__.py
a = 1

$ cat  new_project/tests/test_simple_package.py
from simple_package import a

def test_a():
    assert a == 1

I see the following error message:

  • In the pytest Test Explorer sidebar (left hand side, beaker icon): pytest Discovery Error [pytest-subdirectory-test] Show output to view error logs
  • In the output: E ModuleNotFoundError: No module named 'simple_package'

Is it possible to configure pytest to work with such a repository structure?

It would seem strange that this wouldn’t work, because many projects would likely be a mix of different code written in different programming languages. Therefore it would seem strange if the VS Code pytest integration can only be made to work in the most simple case where all the Python packages are at the top level of the repository.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    There is (what I personally feel is) a "hack" to make this work. It involves modifying the PYTHONPATH, from within code, from within the test source files.

    import sys
import os
sys.path.append(os.path.abspath('./new_group')

    Here's why I don't like this:

    • I have never been of the opinion that modifying the PYTHONPATH is a good solution to any problem
    • I don't like it because it adds in a dependency which really shouldn't be there
    • In this case, I consider it to be somewhat acceptable since the only additional dependency is confined to within the test code and not the source code
    • However, it still has problems. Most notably, every new test_x.py test file needs to have this same hack as the first few lines
    • This results in breaking the DRY principle, since we need this repeated code in many files. Because we have resorted to copy and paste, this becomes more difficult to maintain
    • There is no automatic way of updating the import statements. Usually, if we move files and folders around, VS Code and the Python plugin do a reasonable job of updating the imports for us. There is no way for the VS Code Python plugin to do this if we manually make changes to sys.path

  2. 0

    There is another solution, which again, is a bit of a hack, and again, I don’t really like it much. It involves making everything into a single package.

    Here’s how we do that:

    Add __init__.py to new_project and old_project. This converts both directorys into Python Packages. Because these packages exist at the top level, we can import from them without having to modify PYTHONPATH.

    The code in the tests needs to change slightly. The import statement becomes:

    from new_project.simple_package import a

    Unlike my previous solution (hack), this has the advantage that VS Code/Python plugin can automatically update the imports if we move files and folders around, or rename things. (Under some, but not all conditions.)

    However I still don’t like this. In particular, there are some potential problems:

    • I don’t know how this would affect deployment, or building, of packages (if someone knows please let me know). I would guess at the very least, we will end up with one additional level of namespace
    • The additional level of namespace is pointless, and potentially misleading or confusing

    In addition, I don’t this this really solves the problem. I think we need to move the test folders to the top level as well (not 100% sure on this)

    If so we end up with this structure:

    new_project/
    __init__.py
    ... other
old_project/
    __init__.py
    ... other
tests/
    test_new_project/
        ... tests
    test_old_project/
        ... tests

    This maybe isn’t the end of the world, since the tests are still separated (from the source). But the fact that they are bundled together in the same subdirectory is potentially an issue because we can’t split the project into two independent parts, whereas if the two top level directories were new_project and old_project these would truly be two independent source trees which we could split into two independent repositories.

    You may ask: Since I propose to make both new_project and old_project into Packages, why bother with these at all. Just keep all the previously existing packages at the top level.

    The issue with this is for a complex project with many packages, there probably won’t be an obvious way to tell which packages belong to which project.

    Consider this example:

    mem_cache/
mem_cache_webserver/
mem_cache_cli/
data/
test_mem_cache/
expected_output/
output/
input/
test/
old_memcache/
util_io/
memcache_simulation_driver/
memcache_simulation/

    Which of these are part of the new project and which belong to the old project? There is no way to tell.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search