Description »

Current state-of-the-art image segmentation approaches use convolutional decoders to represent objects [1]. Inspite of their success, even when representing simple shapes, the standard decoders involve computing many intermediate feature representation (at different resolutions) and consist of 1M+ parameters. This results in siginificant increase in resource consumption (computation and GPU memory), specially when working with image volumes. As an alternative to convolutional decoders, we would like to explore the possibility of an alternative object representation technique in this project.

The idea is inspired by the Fourier transform in which we try to approximate a signal as a weighted sum of basis functions. Based on this idea, [2] use weighted sum of spherical harmonic functions to represent 3D objects. Extending the idea, [3, 4] uses improved version of the same representation. For instance, [3] demonstrate how an Amygdala (in human brain) can be approximated as a weighted sum of basis functions (see Fig. 1).

The project has two steps,

• Perform spherical harmonic transform on two datasets and identify the number of basis functions required to represent the object with sufficient accuracy.
• Propose an encoder architecture that can predict coefficients (weights) used in the weighted sum of basis functions from input image volumes.

Datasets:

• CT images of Liver.
• Electron Microscopy images of Synaptic junctions.

References:

• [1] O. Ronneberger and T. Brox. “U-Net: Convolutional Networks for Biomedical Image Segmentation”, MICCAI 2015
• [2] L. Shen and H. Farid. "Modeling three-dimensional morphological structures using spherical harmonics", Evolution 2009
• [3] Cheng-Jin and T. Bretschneider. “Local Shape Representation in 3D: from Weighted Spherical Harmonics to Spherical Wavelets”, BMVC 2012
• [4] Cheng-Jin and T. Bretschneider. “Shape Retrieval using 3D Zernike Descriptors", BMVC 2012

Back to the project list.