Training OverlapSegmentationNet with gpu acceleration on a Ubuntu powered laptop.

Set-up

Harnessing the gpu 1 : nvidia driver

It is  supposed that a fresh ubuntu 16.04 is set-up.

Install Nvidia proprietary driver from the system parametres menu:

The NVidia installed files, with synaptic package manager, are:

Nvidia acceleration can be switched on graphically after having installed nvidia-settings :

 Check driver installation from a terminal by starting glxgears:

When NVIDIA mode is selected, glxgears performances should increase.

Harnessing the gpu 2: install CUDA 8, cudnn 5.1

CUDA8 was installed from a .deb file available from nvidia's site:

Check cuda installation using the "./deviceQuerry" method (see the end of the post).

The Cudnn 5.1 library was installed by copying the files at the right places (Installing cudnn 6 from .deb archive wasn't a good idea. )

 

Install Tensorflow and Keras 

There's different ways to install tensorflow. Here, both tensorflow with gpu support and keras were installed  in a virtual environnement with pip.

Tensorflow 1.2.1 and Keras 2.0.6 were installed .

h5py was installed from pip too.

Don't forget at the end from a terminal to :

$source .bashrc

Installation of OverlapSegmentationNet:

OverlapSegmentationNet is an implementation of UNet by Dr Hu .

Download image-segmentation-chromosomes or clone it from repository with git.

The image-segmentation-chromosomes directory contains:

$ ls
code  images    README.md

Dataset:

The first version of the low resolution dataset was used. Download it and move it into the code directory:

$ ls

create_histogram_visualisation.ipynb  processInputImages.py~

Explore-data.ipynb                    __pycache__

Explore-model2.ipynb                  sub-trainModel.py

histogram.png                         trainModel.py

LowRes_13434_overlapping_pairs.h5     trainModel.py~

models                                utilities.py

OverlapSegmentationNet.py             utilities.py~

OverlapSegmentationNet.pyc            utilities.pyc

preprocessing-jp.ipynb                xdata_88x88.npy

processInputImages.py                 ydata_88x88_0123_onehot.npy

In a terminal,  run processInputImages.py as follow:

$python processInputImages.py

The code will generate the two files highlighted in green above.

Training

Switch into gpu mode with (then logout / login):

 

Activate the proper virtual environnement. Mine was called tfgpu, so for my computer, it is:

$source VirtualEnv/tfgpu/bin/activate

Edit the code to set-up the desired number of epoch

From a terminal, run trainModel.py. For example in my computer, it is:

(tfgpu) jeanpat@WA50SHQ:~/image_segmentation_chromosomes/code$ python trainModel.py

If the gpu is properly used by tensorflow the terminal should yield something like:

The usage of the gpu is mentionned, furthermore the cpu usage is not stucked on 100%.

On a modest nvidia GT740M gpu, 14 epochs were run for 8~9 hours.