Ubuntu – How do I get Vim to 're'source my virtualenv while in it, without closing all my buffers?

The purpose of this command:

$ source .venv/bin/activate

is to change a bunch of environment variables for the current process and its eventual children, the most important being $PATH.

Those changes are carried over when you fork a new process (Vim, the Python REPL, a sub-shell, etc.) but the parent environment is typically copied into the child environment, making the two environments completely separate. Changing the parent environment has no effect on the child environment and vice-versa.

Activating a Python virtual environment is not something you do often and especially not something you do after installing a package, unless the package somehow requires it, which should be very rare and which is not the case with Pandas to begin with.

Now, when you do the following in Vim:

:!pip install pandas

you effectively:

1. Create a new sub-shell that inherits the parent’s virtual environment.
2. Install pandas in that virtual environment.
3. Quit that sub-shell.

At no point did anything change in the child environment or in the parent environment. All that happened is that a bunch of files were added in a bunch of directories under .venv/. There is no need to "re-source" anything, so doing:

:!source .venv/bin/activate

in Vim serves no purpose whatsoever, as…

• it does nothing package-specific,
• whatever changes it does — if any — to the sub-shell’s environment dies with it,
• it can’t be expected to do anything to Vim’s environment or to its parent anyway.

Now let’s go back to the initial, under-specified, problem:

1. You install Pandas in the current virtual environment.
2. Something complains that Pandas isn’t installed.

I went trough the following steps in order to get a sense of what you are dealing with:

1. Create a dummy project and cd into it:

   $ mkdir testpython
   $ cd $_
   

2. Create a virtual environment:

   $ python3 -m venv .venv
   

3. Activate it:

   $ source .venv/bin/activate
   

4. Open a dummy Python file in Vim:

   $ vim foo.py
   

5. Put some dummy code in foo.py:

   import pandas as pd
   s = pd.Series([1, 2, 3, 4, 5])
   print(s)
   

6. Test it without Pandas installed:

   Traceback (most recent call last):
     File "/Users/romlafou/Documents/foopython/foo.py", line 1, in <module>
       import pandas as pd
   ModuleNotFoundError: No module named 'pandas'
   
   shell returned 1
   

   Well yeah, Pandas is not installed.

7. Install Pandas from Vim:

   :!pip install pandas
   

8. Run the buffer through the interpreter:

   :w !python3
   0    1
   1    2
   2    3
   3    4
   4    5
   dtype: int64
   

   That’s it. Vim doesn’t complain, Python doesn’t complain, the shell doesn’t complain, the terminal emulator doesn’t complain… Everything is fine.

FWIW, I also started a REPL in Vim’s terminal with :term python3 and everything worked flawlessly. No complaint whatsoever about Pandas not being installed.

To sum it up:

• a "virtual environment" is really a bunch of environment variables,
• when you start Vim, it inherits the virtual environment of its parent process
• installing a package in the current virtual environment is not expected to change the virtual environment, whether it is installed from within Vim or not,
• the package is immediately available to any python sub-process (!python3 %, :w !python3, :term python3).

Which begs the question: what is complaining about missing Pandas after you have installed it. To which we still have no answer.

My hypothesis is that you are trying to do something like:

:py3file %

which indeed throws an ugly error:

Traceback (most recent call last):
  File "", line 1, in <module>
  File "foo.py", line 1, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'

which is to be expected because the Python interpreter used by Vim under the hood is very unlikely to be the one in your .venv/bin/ and it is very unlikely to be aware of any project-specific virtual environment.

If you absolutely insist on using the :py* family of commands, then you should probably head off to :help if_pyth.txt and try to work something out.

But I wouldn’t recommend going that path as the other methods, :[range]w !python3 being my favorite, all honor your virtual environments out of the box.