- About virtualenv
- Combine virtualenv with IPython
- Proof by trial
- Packages and version
I recently got introduced to virtualenv: a “tool to create isolated Python evironments”. It allows to have a fine grain control on the dependencies of each of your python project, and separate each project environment from the others.
For example, if a cool project requires the
spamspam v1.1 and another cool project requires the version 1.2, performing a system-wide update of the
spamspam library from 1.1 to 1.2 would cause the first project to break. To avoid this kind of problem, you can place each project in a separate virtualenv, and force the version of the
spamspam library to the required one.
You can read more about virtualenv on their PyPI page, but a important takeaway is that you will have pseudo-root rights in your virtualenv: you will be able to install whatever python packages you need. I strongly recommand to use pip for that.
Combine virtualenv with IPython
If you don’t know IPython, check it out. It adds powerful features like autocompletion, colors, crazy data visualisation tools, help, source code visualisation of packages and functions, etc. It’s just raw awesomness.
Now, the problem is that virtualenv default behavior is to ignore all packages located in your
/usr/lib/pythonX.X/site-packages directory when creating a new virtualenv, which is precisely where IPython is installed. You thus won’t be able to use IPython in your virtualenv without a little bit of hacking.
There are two ways to make it work:
- the good way
- the bad way
I’ll describe both techniques and you’ll decide which is which.
Install a local IPython in each virtualenv
First, you can install IPython in each new virtualenv, with
$ pip install ipython
That will work well, but the package is quite heavy (~10Mb), and frankly, that could/should be automaded. Another problem I see with this technique is that you might want to push your code on a repository, for other people to be able to contribute. If so, you might want to push the virtualenv content, to simplify them the set-up work. Even if extremely handy, IPython might not be vital for you project, and you should only install required packages in the virtualenv, for the sake of testing.
One IPython to rule them all
Now there’s a trickier solution: we can configure your system IPython, to make it aware of your virtualenv.
This has been extremely well done by Chris Lasher, and all instructions and explanations can be found
The main idea is to check if you’re running in a virtual environment when you launch IPython. If so, add all your virtualenv python environment at the begining of your
I however encountered a small problem with this method. IPython being in
/usr/lib/python2.7/site-packages directory, my virtualenv was not even aware of an IPython module being installed, and this error was fired even before executing the code shown in Chris’ cool blogpost:
Traceback (most recent call last): File "/usr/bin/ipython", line 30, in <module> import IPython.Shell ImportError: No module named IPython.Shell
Here’s the small hack I did to make everything work.
First I added these lines to my
/usr/bin/ipython file, before importing
import sys if "/usr/lib/python2.7/dist-packages" not in sys.path: sys.path.append("/usr/lib/python2.7/dist-packages")
append command will only be executed when I’m launching IPython from a virtualenv, otherwise, the
if statement would not be
/usr/lib/python2.7/dist-packages is in my
sys.path, I can import
IPython.Shell from my virtualenv.
The code contained in
~/.ipython/virtualenv.py (as described in Chris’ article) will now be executed but still need a “counter-hack”,
to cancel the effects of what we wrote in
/usr/bin/ipython. We need to completely remove
/usr/lib/python2.7/dist-packages from our
To do that, we’ll add a small snippet in
for item in sys.path: if '/usr/lib/python2.7/dist-packages' in item: sys.path.remove(item)
Here is the final
import site from os import environ from os.path import join import sys if 'VIRTUAL_ENV' in environ: virtual_env = join(environ.get('VIRTUAL_ENV'), 'lib', 'python%d.%d' % sys.version_info[:2], 'site-packages') # Remember original sys.path. prev_sys_path = list(sys.path) site.addsitedir(virtual_env) # Personal hack # Remove /usr/lib/python2.7/dist-packages and associates from the path # Counter the effects of this code, in /usr/bin/ipython # import sys # if "/usr/lib/python2.7/dist-packages" not in sys.path: # sys.path.append("/usr/lib/python2.7/dist-packages") for item in sys.path: if '/usr/lib/python2.7/dist-packages' in item: sys.path.remove(item) # Reorder sys.path so new directories at the front. new_sys_path =  for item in list(sys.path): if item not in prev_sys_path: new_sys_path.append(item) sys.path.remove(item) sys.path[1:1] = new_sys_path print 'VIRTUAL_ENV ->', virtual_env del virtual_env
Proof by trial
Now, let’s try it, to check everything’s working:
balto $ cd path/to/virtualenv/project balto $ source bin/activate (project)balto $ ipython VIRTUAL_ENV -> /path/to/virtualenv/project/lib/python2.7/site-packages Python 2.7.2+ (default, Oct 4 2011, 20:06:09) Type "copyright", "credits" or "license" for more information. IPython 0.10.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In :
Yay. Now, you won’t have to worry about installing IPython when creating a new virtualenv.
To check that everything is working properly, install a package in a fresh virtualenv that you have not installed in your “system” python.
Open two IPython interpreters: one in the virtualenv, one outside, and try to import this newly installed package in both.
The virtualenv IPython should be fine, and the second should fire an
Packages and version
I’m running IPython 0.10.2, Python 2.7.2+, and virtualenv 126.96.36.199.