Documenting the steps how to setup Theano to run on GPU on Ubuntu 14.04 so I can refer to this in the future. With this successfully installed, you can run Keras, convnet, Theano, etc properly.

Why This Article?

Have you ever met those painful issues just trying to setup your GPU for scientific programming:

  • Getting a blinking cursor after installing the “latest” Nvidia drivers?
  • Getting a blinking cursor after installing the “latest” CUDA toolkit?
  • Login loop in Ubuntu login screen?
  • NVRM: rm_init_adapter(0) failed?
  • NVIDIA: could not open the device file /dev/nvidia0 (input/output error)
  • …

If yes, then read on.

I cannot guarantee your OS does not break after running the steps below, so please backup everything, run as your own risk.

You may want to read the Things That Does Not Work For Me part below first before following the instructions.

Specifications

My server has the following specifications:

  • OS: Ubuntu 14.04 LTS, X64
  • GPU: Nvidia Geforce GTX 780
  • Fresh install of Ubuntu 14.04 LTS

Instructions

If you fail in the middle of any of this steps, you are advised to restart and follow the steps carefully again. Use common sense. and if you don’t know how to fix it, too bad,

  1. Download latest (or very recent) Nvidia display driver for Linux:
wget http://us.download.nvidia.com/XFree86/Linux-x86_64/352.63/NVIDIA-Linux-x86_64-352.63.run

2. Download latest (or very recent) CUDA Installation Run file for Ubuntu 14.04:

wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run

3. Get into recovery mode to install the files, because you cannot install your display driver with X server on.

reboot

  1. Once you are in GRUB, select “Advanced Options for Ubuntu”.

  2. In the recovery mode, select “network”, press yes. Once the networking has been enabled, this will also set your drive to read/write mode.

6. In the recovery mode, select “root” to get a root prompt.

  1. Go to where you downloaded the previous files at, and run NVIDIA-Linux-x86_64-352.63.run .
chmod +x NVIDIA-Linux-x86_64-352.63.run # Make it executable

./NVIDIA-Linux-x86_64-352.63.run # Install Nvidia display driver

For yes/no questions, take “yes”. For locations, press “enter” to take default values.

  1. Setup your environment variables.

Make sure your LD_LIBRARY_PATH, LIBRARY_PATH and PATH includes the places it needs to include:

# ~/.bashrc
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-7.5/lib64:
export LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-7.5/lib64:
export PATH=$PATH:/usr/local/cuda-7.5/bin

Don’t forget to:

$ source ~./.bashrc
  1. Run nvidia-smi to confirm that your setup is correct.
nvidia-smi

nvidia-smi should be on your path already, so this should work. If it doesn’t:

  • You must have done something wrong, better go back to step 1. and do it all over again.
  • You can see if it’s at /usr/bin/nvidia-smi .

You should then see something similar to this:

Mon Nov 23 21:01:17 2015 
+------------------------------------------------------+ 
| NVIDIA-SMI 352.39 Driver Version: 352.39 | 
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 780 Off | 0000:01:00.0 N/A | N/A |
| 34% 33C P0 N/A / N/A | 137MiB / 3068MiB | N/A Default |
+-------------------------------+----------------------+----------------------+
 
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 Not Supported |
+-----------------------------------------------------------------------------+
  1. Go to where you downloaded the previous files at, and run cuda_7.5.18_linux.run .
chmod +x cuda_7.5.18_linux.run # Make it executable

./cuda_7.5.18_linux.run # Install Nvidia display driver

For yes/no questions, take “yes”. For locations, press “enter” to take default values.

  1. Re-run nvidia-smi just to be sure everything is still working well.
nvidia-smi

You should then (again) see something similar to this:

Mon Nov 23 21:08:56 2015 
+------------------------------------------------------+ 
| NVIDIA-SMI 352.39 Driver Version: 352.39 | 
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 780 Off | 0000:01:00.0 N/A | N/A |
| 34% 28C P8 N/A / N/A | 271MiB / 3068MiB | N/A Default |
+-------------------------------+----------------------+----------------------+
 
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 Not Supported |
+-----------------------------------------------------------------------------+

11. Reboot now back to desktop.

reboot
  1. Time to install Theano and python, basically the first part of this guide. Run the following commands:
sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git
sudo pip install Theano
  1. Copy and paste the below code into check1.py. Now we need to make sure GPU is indeed working. I am copying the instructions from deeplearning.net.
from theano import function, config, shared, sandbox
import theano.tensor as T
import numpy
import time

vlen = 10 * 30 * 768  # 10 x #cores x # threads per core
iters = 1000

rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
for i in xrange(iters):
    r = f()
t1 = time.time()
print("Looping %d times took %f seconds" % (iters, t1 - t0))
print("Result is %s" % (r,))
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
    print('Used the cpu')
else:
    print('Used the gpu')
  1. Run the check1.py to confirm that Theano is working correctly with CPU.
THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python check1.py

This should give:

[Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]
Looping 1000 times took 1.999963 seconds
Result is [ 1.23178029 1.61879337 1.52278066 ..., 2.20771813 2.29967761
 1.62323284]
Used the cpu
  1. Run the check.py to confirm that Theano is working correctly with GPU.
THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python check1.py

This should give:

Using gpu device 0: GeForce GTX 780
[GpuElemwise{exp,no_inplace}(<CudaNdarrayType(float32, vector)>), HostFromGpu(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 0.583838 seconds
Result is [ 1.23178029 1.61879349 1.52278066 ..., 2.20771813 2.29967761
 1.62323296]
Used the gpu

Note that: computation time has decreased, and it is indeed using GPU now.

  1. Done! If you reached here and did not meet any issues, I hope you can comment and confirm that this does help you, so others can trust this guide. If you have still spent hours/days, blame Linux. =]

Things That Does Not Work For Me

  1. Installing current nvidia drivers from apt-get.
$ apt-get install nvidia-current

Normally you would think apt-get install the latest nvidia drivers would be the best to go. Nope, after the installation 1) nvidia-smi does not work 2) I cannot get back into desktop 3) get error messages like RmInitAdapter failed in dmesg 4) no monitor found in /var/log/Xorg.0.log

  1. Install CUDA from apt-get.
$ apt-get install nvidia-cuda-toolkit

The same as 1., equally disastrous: 1) nvidia-smi does not work 2) I cannot get back into desktop 3) get error messages like RmInitAdapter failed in dmesg 4) no monitor found in /var/log/Xorg.0.log

3. Installing current nvidia from another “defacto” repository.

sudo apt-add-repository ppa:ubuntu-x-swat/x-updates
sudo apt-get update
sudo apt-get install nvidia-current

The same as 1., equally disastrous: 1) nvidia-smi does not work 2) I cannot get back into desktop 3) get error messages like RmInitAdapter failed in dmesg 4) no monitor found in /var/log/Xorg.0.log

4. Anything about Bumblebee.

Not sure what this is, just not related. Forget about it.

5. Also, I don’t have advice trying to repair a broken Ubuntu if you did anything of the above. But maybe you can try removing nvidia drivers and reinstalling the desktop, but doesn’t work for me either.

apt-get remove --purge nvidia*
apt-get install --reinstall ubuntu-desktop unity
apt-get install nvidia-current
  1. Using any nvidia driver like 340 or 304 version. They do not work.

The only thing that works for me is a fresh reinstall of Ubuntu 14.04. Nothing else.

Final Words

The installation guide from DeepLearning.net is very helpful, but it contains poison advice that messed up my display of my OS completely.

BTW, ever heard of “Linux is user-friendly but it is very selective of its friends.”? I know I am certainly not its friend… 😉