Friday, August 25, 2017

Training OverlapSegmentationNet with gpu acceleration on a Ubuntu powered laptop.


Harnessing the gpu 1 : nvidia driver

It is  supposed that a fresh ubuntu 16.04 is set-up.
Install Nvidia proprietary driver from the system parametres menu:
The NVidia installed files, with synaptic package manager, are:

Nvidia acceleration can be switched on graphically after having installed nvidia-settings :

 Check driver installation from a terminal by starting glxgears:

When NVIDIA mode is selected, glxgears performances should increase.

Harnessing the gpu 2: install CUDA 8, cudnn 5.1

CUDA8 was installed from a .deb file available from nvidia's site:

Check cuda installation using the "./deviceQuerry" method (see the end of the post).

The Cudnn 5.1 library was installed by copying the files at the right places (Installing cudnn 6 from .deb archive wasn't a good idea. )


Install Tensorflow and Keras 

There's different ways to install tensorflow. Here, both tensorflow with gpu support and keras were installed  in a virtual environnement with pip.

Tensorflow 1.2.1 and Keras 2.0.6 were installed .

h5py was installed from pip too.
Don't forget at the end from a terminal to :

$source .bashrc

Installation of OverlapSegmentationNet:

OverlapSegmentationNet is an implementation of UNet by Dr Hu .

The image-segmentation-chromosomes directory contains:
$ ls
code  images


The first version of the low resolution dataset was used. Download it and move it into the code directory:
$ ls
Explore-data.ipynb                    __pycache__
OverlapSegmentationNet.pyc            utilities.pyc
preprocessing-jp.ipynb                xdata_88x88.npy                 ydata_88x88_0123_onehot.npy

In a terminal,  run as follow:
The code will generate the two files highlighted in green above.


Switch into gpu mode with (then logout / login):

Activate the proper virtual environnement. Mine was called tfgpu, so for my computer, it is:
$source VirtualEnv/tfgpu/bin/activate
Edit the code to set-up the desired number of epoch
From a terminal, run For example in my computer, it is:

(tfgpu) jeanpat@WA50SHQ:~/image_segmentation_chromosomes/code$ python
If the gpu is properly used by tensorflow the terminal should yield something like:

The usage of the gpu is mentionned, furthermore the cpu usage is not stucked on 100%.

On a modest nvidia GT740M gpu, 14 epochs were run for 8~9 hours.