Installing MayaVi for Python 3.5 in a virtual env

Hi everyone,

this might not be my typical post. But if you have a problem with that, well… fuck you. A few days back I installed MayaVi in a virtual environment that is running Python 3.5 and configured it to run from my Jupyter notebooks. This was a huge pain, but I managed to get it to work (major depression obtained during that process). Of course in the end I had no idea what made it work, but I was happy. Fast forward to today and I need to redo this since I decided to recreate my virtual env on my new machine. A GREAT (semi-maintained) guide of how to set up such a virtual env can be found here here. Since I am doing this again, I decided to document it this time to maybe save you the depression of figuring this whole damn thing out. All in all it shouldn’t take more than a few hours.

Prelimary stuff

Big warning: yes I studied computer science, but I am, as with most things in life, just very mediocre. I will find my way around, call things by the wrong name and eventually get stuff to work. So will the following be the cleanest possible solution? Probably not. There are some dirty hacks, but in the end MayaVi runs. The only reason I am writing this, is that I found tons of different blog posts all telling you how to do this, from somewhere between 2000 and 2004. Somewhat outdated :D If anybody just saves a little of time reading this, then it was worth it. Likely, it will be a future me doing this again at some point.

Here we will be aiming to installing MayaVi from PyPI, but with a properly working backend that we built ourselves. I’m installing this on Ubuntu 14.04 with quite some packages pre-installed and I’m using a horribly configured zsh. So some stuff might need to be slightly changed if you are using another shell or Linux version. I guess, if you are messing with this kind of stuff though, you ought to be able to fix your things. And obviously these guides only seem to have a rather limited lifespan. The moment stuff changes, everything could break. If you are reading this in 2020, just accept that you will to figure it out yourself. From here on out I will assume that you have setup stuff as described in the above mentioned guide. But it pretty much just boils down to a local gimmick of your global root with a usr, lib, include etc. folder located at ~/inst/. Furthermore, we are talking about how to do it for a virtual env. So let’s say I will assume your virtual env root is located under~/VIRT_ENV/. Change these two things throughout the following commands to your paths.

First shot at installing MayaVi!

So in theory installing MayaVi should be as easy as:

pip install mayavi

It will however present you with the following Error:

 ImportError: No module named 'vtk'

Now this can have two reasons, one might be that everything should work, but you just don’t use the site-packages in your virtual environment, or your (possibly) installed vtk doesn’t have Python 3 bindings. If it doesn’t present you with this error, be happy and go live your life. Or maybe you should check if you are actually using Python 3.

VTK

So let’s build VTK ourselves and install it locally so that Python can use it.

cd ~/inst/src
git clone https://github.com/Kitware/VTK.git
mkdir build-VTK
cd build-VTK
ccmake -D CMAKE_BUILD_TYPE=Release \
       -D CMAKE_INSTALL_PREFIX=~/inst \
       -D VTK_WRAP_PYTHON=ON \
       -D VTK_PYTHON_VERSION=3 \
       -D VTK_INSTALL_PYTHON_MODULE_DIR=~/VIRT_ENV/lib/python3.5/site-packages \
       -D PYTHON_EXECUTABLE=~/VIRT_ENV/bin/python3.5 \
       ../VTK

Hit c several times to configure and then g to generate then finish with a make and a make install. The make will download a lot of data from a somewhat slow server as it seems (~200MB) and the building also takes quite a while here. Go make yourself a vegan sandwich and come back afterwards. This will install everything into your local ~/inst folder. The problem is however, that Python expects the library file location to be in your path. If you like me don’t have your local installation folder in your path and library path, then this will cause problems. There are several solutions:

1. You can add them to the correct paths in your shell config file (such as .bashrc or .zshrc). This will fix it, however it will affect everything you do, not only when you do something with your virtual env.
2. You can change the above cmake command to install it purely to the virtual env. This will however not allow you to use it from anywhere else.
3. You make some links between the library files of VTK in your local install folder and the place where they are searched for in when you import vtk.

I’m going for the last option. So let’s create the links with the following command:

for x in ~/inst/lib/libvtk*.so.1; do ln -s $x ${x/~\/inst/~\/VIRT_ENV}; done

Cool kids would now run vtk tests, however, I don’t want to do that. We should however make sure that we can at least import VTK! With your virtual env activated, run:

python -c 'import vtk; print(vtk.VTK_VERSION)'

This should print out something like 7.1.0. Check that the version you get is actually the one you just built yourself and not some old system version.

Second shot at installing MayaVi!

So with VTK fixed, let’s try again.

pip install mayavi

Success! At least for me this now works. Are we done. I don’t think so just yet. Open up a notebook or iPython console and run the following code:

import numpy as np
from mayavi import mlab

t = np.linspace(0, 4 * np.pi, 50)
x = np.sin(2 * t)
y = np.cos(t)
z = np.cos(2 * t)
s = 1 + np.sin(t)

mlab.points3d(x, y, z, s)

This will present you with another error, key of which is:

ImportError: Could not import backend for traits

Now after playing around with all of this stuff. I still couldn’t be bothered to read up on what traits exactly is or does, but I know we need it to run MayaVi. Traits can use two kinds of gui backends, wxPython or PyQt. Since wxPython seems pretty dead and doesn’t support newer Python versions, we have to go with PyQt. PyQt is just a wrapper around Qt and it has two (well kind of three) versions. There is PyQt4, PyQt5 and the kind of third version PySide. All of these could, in theory, be used as a backend, but traits doesn’t support PyQt5 (which would directly be available from PyPI) and PySide doesn’t work for Python 3.5 (but is also in PyPI). So we are left with PyQt4 as our only option.

PyQt4

So how do we get this? Yes, we have to build it ourselves. Ironically, even though there is a PyQt4 and PyQt5, PyQt4 will wrap either Qt4 or Qt5, I’m not sure who picked the naming schemes here. I was very confused when it started doing that. Now my system has a bunch of Qt5 libraries preinstalled and I can globally install certain libraries myself. If you don’t have sudo rights on your machine and you are missing relevant Qt4/5 libraries, you will have to install that yourself too. I didn’t do that, so you venture there yourself and fall of the edge of the world. Bye.

Now you can get a fairly good guide of how to install PyQt4 here. Apparently before installing PyQt4, you will fill have to install and build SIP. I don’t know what that is and frankly I don’t care. Let’s build it.

SIP

They only have mercurial repo, stuff I no can do, and a sourceforge link, durpadurp. So go here and get the latest version and extract it in your `~/inst/src’ folder. Now let’s build it, which is rather straight forward. From your activated virtual env do.

cd ~/inst/src/sip-4.18
python configure.py
make
make install

Accept the terms of the license by typing yes (no y won’t suffice). This will install sip into your virtual env. Since it is easy to reproduce for every new virtual env, I will not bother changing paths to try and install it in the ~/inst folder.

Actuall PyQt4 installation

So, almost there! You have to download another archive from sourceforge. Unzip it next to your SIP folder and then run the following:

cd ~/inst/src/PyQt-x11-gpl-4.11.4
python configure-ng.py

Note: the configure-ng.py instead of the configure.py. They say this is newer and better and fancier, let’s just believe them. Now you have to pay close attention to what happens. At some point the configuration says something like:

These PyQt4 modules will be built: QtCore, QtGui, QtNetwork, QtXml, QtOpenGL,
QtSql, QtTest, QtDBus.

This is the list of Qt libraries that will actually be wrapped by PyQt4. I never actually installed Qt5, however, apparently some of the software I use depends on Qt5, thus parts of the libraries have been installed globally. In order for traits and thus in turn MayaVi to work, your PyQt4 needs to wrap at least QtCore, QtGui and QtSvg. At least these are the hard dependencies I have found so far. As you can see my PyQt4 configuration isn’t planning to wrap QtSvg. It took me quite some time to figure this out and then some more time to figure out why. I simply didn’t have it installed. Here is the part where I pity you if you don’t have sudo rights, because have fun installing Qt5 from source. (Although it might not even be such a pain, who knows!) Instead I do:

sudo apt-get install libqt5svg5-dev

Now if we run the configuration script again, you should now notices that the QtSvg module will be built! Yay! So continue by running the following, as always while having your virtual env activated:

make
make install

This again installs it in your virtual env. And this concludes installing stuff!

Testing MayaVi!

We should now be able to test MayaVi again! OMG! Such Excite! For vanilla Python, MayaVi should now be working. There is one more thing that needs to be done if you want to use it with iPython or in a Jupyter notebook. Every time you start either of these you need to do the following before you call something: %gui qt. This will tell iPython to use qt as a gui backend. Maybe you can also set this in some config file, but that is up to you to find out. If you run the above testing block with this command prepended, you should see some window popping up with some 3D balls in a spiral. It’s important that you can drag stuff around with your mouse, without the window freezing. If this is not the case then something is still wrong. So to test it, activate your virtual env and run the following:

ipython
%gui qt
import numpy as np
from mayavi import mlab

t = np.linspace(0, 4 * np.pi, 50)
x = np.sin(2 * t)
y = np.cos(t)
z = np.cos(2 * t)
s = 1 + np.sin(t)

mlab.points3d(x, y, z, s)

If everything works well, that’s it. If not, too bad :( But it should be working, so now you can use MayaVi in whatever way you want to. Congratulations and it case this guide helped, you’re fucking welcome, next time you buy lunch think of this moment and eat something without animal cruelty.