Sunday, February 18, 2018

Training OverlapSegmentationNet with GPU acceleration on google-colaboratory

 OverlapSegmentationNet is a python implementation of Unet intented to resolve pairs of overlapping chromosomes by semantic segmentation (See previous post).

OverlapSegmentationNet can now be trained on the google colaboratory platform with GPU acceleration.

The whole files and data are available from google drive. Wouldn't have been able to set-up without the following article.

Thursday, November 16, 2017

No more decent computing mean

Laptop "molten", no computing capabilities at home for a while (appart from a those of a pavillion amd A4-1200 APU; too short to train a nn).

Sponsoring welcome.

Thursday, October 12, 2017

Predictions from OverlapSegmentationNet after 14 epochs

OverlapSegmentationNet is a UNet model implemented with Keras. The network was trained for 14 epochs (~8 hours on a GT740M GPU), this is not enough to make a good prediction but it is enough to play with the model to understand how a prediction from a single image looks like.


The dataset used for training was the first proposed on kaggle. It consists in 13434 pairs of greyscale/ground truth images of size 88x88. Each greyscaled image has a pair of overlapping chromosomes. The overlaps were synthetically generated from images of single chromosomes:

Making a prediction from a single greyscaled image:

The aim is to predict the segmentation labels with a Keras model (OverlapSegmentationNet). The 88x88 greyscaled image must be converted into a 4D numpy array (1, 88, 88, 1).
A prediction is then 4D array of shape ( 1,88,88,4), since there's one component for each label (background, chromosome 1, chromosome2, overlap). A prediction looks like:
where each component is a floating-point image. With another chromosome, we have:
A prediction takes ~ 40 ms for one low resolution image.

Prediction made on an image which do not belong to the train dataset:

A prediction was made on an image belonging to a validation dataset:
To be used for prediction, the image was down sampled and recut to fit 88x88 (and its corresponding ground truth segmentation too):
Clearly, with so few epochs the prediction is bad:
The one-hot encoded prediction can be thresholded and back converted into a multilabel segmentation and compared to the ground truth segmentation:

Notebook is here:

More details are available in the notebook:


Friday, August 25, 2017

Training OverlapSegmentationNet with gpu acceleration on a Ubuntu powered laptop.


Harnessing the gpu 1 : nvidia driver

It is  supposed that a fresh ubuntu 16.04 is set-up.
Install Nvidia proprietary driver from the system parametres menu:
The NVidia installed files, with synaptic package manager, are:

Nvidia acceleration can be switched on graphically after having installed nvidia-settings :

 Check driver installation from a terminal by starting glxgears:

When NVIDIA mode is selected, glxgears performances should increase.

Harnessing the gpu 2: install CUDA 8, cudnn 5.1

CUDA8 was installed from a .deb file available from nvidia's site:

Check cuda installation using the "./deviceQuerry" method (see the end of the post).

The Cudnn 5.1 library was installed by copying the files at the right places (Installing cudnn 6 from .deb archive wasn't a good idea. )


Install Tensorflow and Keras 

There's different ways to install tensorflow. Here, both tensorflow with gpu support and keras were installed  in a virtual environnement with pip.

Tensorflow 1.2.1 and Keras 2.0.6 were installed .

h5py was installed from pip too.
Don't forget at the end from a terminal to :

$source .bashrc

Installation of OverlapSegmentationNet:

OverlapSegmentationNet is an implementation of UNet by Dr Hu .

The image-segmentation-chromosomes directory contains:
$ ls
code  images


The first version of the low resolution dataset was used. Download it and move it into the code directory:
$ ls
Explore-data.ipynb                    __pycache__
OverlapSegmentationNet.pyc            utilities.pyc
preprocessing-jp.ipynb                xdata_88x88.npy                 ydata_88x88_0123_onehot.npy

In a terminal,  run as follow:
The code will generate the two files highlighted in green above.


Switch into gpu mode with (then logout / login):

Activate the proper virtual environnement. Mine was called tfgpu, so for my computer, it is:
$source VirtualEnv/tfgpu/bin/activate
Edit the code to set-up the desired number of epoch
From a terminal, run For example in my computer, it is:

(tfgpu) jeanpat@WA50SHQ:~/image_segmentation_chromosomes/code$ python
If the gpu is properly used by tensorflow the terminal should yield something like:

The usage of the gpu is mentionned, furthermore the cpu usage is not stucked on 100%.

On a modest nvidia GT740M gpu, 14 epochs were run for 8~9 hours.

Saturday, July 1, 2017

Counting impacts on a target (Traces of heavy ions beam) with the help of kmeans clustering.

When studying effect of ionizing radiation (heavy ions with high LET) on cells, the fluence (particules/cm²) of the particles beam must be known. Detectors in the GANIL (Caen, France) particles accelerator can estimate the fluence. However, when irradiating cells, radiobiologists (CEA CIRIL) need to localize the impacts of the ions and to count the number of impact per cell.

The application TRACES (scripts for the aphelion application) was developped: It was capable of recording and saving images from a camera, segmenting and counting traces: 
The application was designed to analyse images as the following:

raw image of traces

Counting impacts  in a python notebook:

Contrary to the TRACES application, the following jupyter notebook can't capture images (It should be possible to do it in a python script with the cv2 module) but it can perform some automatic classification.
Distinguishing traces resulting from one or several impacts is a classification issue which can be explored with unsupervised classifier such k-means clustering: