When Chromosomes Collide: ChatGPT's Cytogenetic Saga

Picture this: you're chilling in your lab, studying chromosomes, when suddenly you're faced with a tangled mess of overlapping chromosomes. You need to untangle them and restore their original images, but how? Fear not, fellow scientists, for ChatGPT is here to save the day - even on your trusty, budget-friendly Dell T5500 with 24 GB RAM and GTX 960 4 GB GPU!

The Chromosome Challenge

Our brave scientists have 23 chromosome images, and they've asked ChatGPT to perform the following tasks:

1. Threshold and normalize each image.
2. Rotate and translate each image to create new augmented samples.
3. Generate random triplets of grayscale images and occlusions by taking the maximum of the pixel values.
4. Keep 100 triplets that meet specific criteria for connected components.

And because we're feeling cheeky, we're allowed to throw in some jokes about their English level and geek slang.

The ChatGPT Solution

First, let's dive into the methods we've used to solve this chromosome conundrum.

Thresholding and Normalizing

We apply a threshold to each image, creating a binary mask. We then calculate the mean and standard deviation of the pixel values within the mask, and normalize the image accordingly.

$\mathrm{normalized}\_\mathrm{image}=\frac{\mathrm{image}-\mathrm{mean}}{\mathrm{std}}$

Here's the Python code for this step:

def preprocess_images(images, threshold):
    preprocessed_images = []
    for image in images:
        mask = (image > threshold).float()
        mean = torch.sum(image * mask) / torch.sum(mask)
        std = torch.sqrt(torch.sum((image - mean) ** 2 * mask) / torch.sum(mask))
        normalized_image = (image - mean) / std
        preprocessed_images.append((normalized_image, mask))
    return preprocessed_images

Rotating and Translating

We rotate each image at 30-degree intervals and apply horizontal and vertical translations, generating a collection of augmented samples.

Here's the Python code for this step:

def augment_images(preprocessed_images, rotation_step, translation_range):
    augmented_images = []
    for normalized_image, mask in preprocessed_images:
        for angle in range(0, 360, rotation_step):
            rotated_image = rotate_image(normalized_image, angle)
            rotated_mask = rotate_image(mask, angle)
            for dx in translation_range:
                for dy in translation_range:
                    translated_image = translate_image(rotated_image, dx, dy)
                    translated_mask = translate_image(rotated_mask, dx, dy)
                    augmented_images.append((translated_image, translated_mask))
    return augmented_images

Generating Triplets and Occlusions

We randomly select triplets of grayscale images and compute the occlusions by taking the maximum of their pixel values. We then perform arithmetic summation and bitwise AND on the masks.

Here's the Python code for this step:

def generate_triplets_and_occlusions(augmented_images, num_triplets):
    triplets = []
    occlusions = []
    for _ in range(num_triplets):
        images_triplet = random.sample(augmented_images, 3)
        occlusion = torch.stack([img for img, _ in images_triplet]).max(dim=0).values
        mask_sum = torch.stack([mask for _, mask in images_triplet]).sum(dim=0)
        mask_and = torch.stack([mask for _, mask in images_triplet]).prod(dim=0)
        triplets.append(images_triplet)
        occlusions.append((occlusion, mask_sum, mask_and))
    return triplets, occlusions

Filtering Valid Triplets

We filter the triplets based on the connected components criteria mentioned earlier, keeping only the valid ones up to a maximum of 100 triplets.

Here's the Python code for this step:

def filter_valid_triplets(triplets, occlusions, max_triplets):
    valid_triplets = []
    valid_occlusions = []
    for triplet, (occlusion, mask_sum, mask_and) in zip(triplets, occlusions):
        connected_components = get_connected_components(mask_and)
        if len(connected_components) == 2 or (len(connected_components) == 1 and mask_sum.max() == 3):
            valid_triplets.append(triplet)
            valid_occlusions.append(occlusion)
            if len(valid_triplets) >= max_triplets:
                break
    return valid_triplets, valid_occlusions

Conclusion

With ChatGPT's help, our scientists can now confidently face the challenges of overlapping chromosomes! By using this dataset with triple occlusions, we can train powerful inpainting models that can assist cytogeneticists in their quest to understand the secrets hidden within chromosomes.

Who would've thought that a humble Dell T5500 with 24 GB RAM and GTX 960 4 GB GPU could uncover the mysteries of our genetic blueprint? Thanks, ChatGPT!

100 overlapping, full resolution, chromosomes isolated from 14 metaphases of human lymphocytes

Metaphasics chromosomes were hybrized with a PNA telomeric probe (CCCTAA-Cy3) and an additional oligo DNA probe (Cy5) for testing purposes.

100 overlapping chromosomes can be found in that dataset (jpp21). The dataset consists of 12 bits raw greyscaled images available on the DeepFISH repository. The images of overlapping chromosomes were isolated and converted into 8bits rgb images:

Sample of 100 overlapping chromosomes (full resolution)

The overlapping images are also available as combination of DAPI and Cy3 components:

Combination DAPI+Cy3 images (inversed greyscale)

• link to the color images
• link to the greyscaled images 
• link to the notebook

Making image processing on SageMathCloud. Extraction of chromosomes from multispectral images.

SageMathCloud is a plateform to do mathematics, it also offers the possibility to use a ipython notebook online. Several python libraries are available: numpy, scipy, scikit-image (0.7, 0.93), mahotas 1.1, pandas ...

Once an account is opened (it's free), a project created, some multispectral images were uploaded in a directory of the project. In that project, an ipython notebook was created, this notebook can be run in a project directly in SageMathCloud.

Removing spurious pixels brought by ndimage.find_objects():

The following model image  is a grey scaled image:

Grey image with a pseudo color map

find_objects() succeeds in extracting the letters:

Extracted particles are not convex, their bounding box can contains pixels from other particles in their neighborhood.

For example the letter A, contain pixels belonging to other segmented (labelled) letters. The image of A contains pixels belonging to the letter r and to the letter B. Those spurious pixels have to be eliminated. find_object() provides the bounding box of the segmented particles in a list. The bouding box index seems to correspond to the particle label, this could be used to erase the spurious pixels:

A python function can be written:

It takes two images as arguments: a grey level  image (or stacked images) and a label image and returns a list of images containing the extracted particles from the grey level image. This implementation is much simpler and faster than a previous one .

 

Chromosomes extraction from a MFISH image:

After having uploaded images on cloud.sagemath, for example:

The five components (DAPI excepted) spectral images can be combined in one color image. There is an additional kar image resulting of the segmentation of the MFISH images :

5 spectral images combined into 3 to make a color (rgb) image (left). kar image (right)

Let's use the kar image to produce a label image so that each chromosome is defined by a single label (or grey level):

label image (right)

Now the extraction of individual chromosomes can be done, from the DAPI image:

or from stacked images:

The kar image identifies the chromosomes:

Let's make galleries of chromosomes

The extracted chromosome images have different size. Let's add borders to the images so that the images have all the same size :



Then let's build two galleries, one with MFISH images and one with inverse DAPI chromosomes, the code is visible in the notebook; it yields:

Left: MFISH chromosomes, Right:inverse DAPI

It remains to auto rotate the chromosome. A draft code, at the end of the ipython notebook  in the SageMathCloud project, tries to explore another method than the previously proposed. It remains also to arrange the chromosome according to their classification.

Download the ipython notebook

Go to the notebook on sagemathcloud