Teaching Microscopes to See - A Conversation with Professor Duane Loh

Professor Duane Loh
 

What are computational lenses?

Humans are able to see the surroundings because eyes detect and measure light emitted from objects in the environment. The brain then interprets the patterns of detected light as visual perception. The same combination holds true in high-resolution microscopy used to see the very small building blocks of the world: atoms, molecules, proteins, etc. Instead of using the eyes and brain, we can use special lenses (for x-rays and electrons) and computers to examine these structures on the very small scale.

In the past, scientists’ abilities to view this microscopic world was largely dependent on how well these lenses were made. In recent years, as computing power becomes cheaper, Professor Loh shares that it is now possible to replace some of these physical lenses in an imaging system with computational optics algorithms to reliably form images of specimens. 

Single particle imaging using x-rays or electrons is an up and coming field of research. The general idea is to use many random incomplete and noisy views of a system, to statistically piece together a higher quality and/or higher dimensional “image” of the myriad processes within this system.

For example, researchers can recover a three-dimensional structure of a protein by capturing the images of many randomly oriented two-dimensional views of copies of this protein in various orientations.  This type of imaging is now routinely done using x-ray and electron-based imaging.

So far, this technique has been successful when used for sets of identical proteins carefully prepared in experiments. The ambitious goal now is to extend this technique to image a mixed or blended set of floppy biomolecules, some of which might be previously unknown, while others might be recognisable but have now adopted new structural states. 

The process of analysing such minute elements is no simple feat. Professor Loh and his team begin the process by understanding the physics behind the contrast mechanism that created these images before studying the actual digital processing carried out on the images. He explains that microscopy images are not merely collections of pixels — but have the ability to capture how the world interacts with light and electrons!

The primary differences between using x-ray diffraction and electron microscopy data, versus looking at light or fluorescence microscopy images can be assigned to two factors – sample damage and the amount of data that can be gathered.

In x-ray diffraction patterns, the pixel values of the original images are all mixed in a so-called incomplete, but fairly noisy. Fourier transformed pattern, as x-ray photons scatter weakly with matter. While this weak interaction makes the mathematics of their interaction easy for researchers to exploit for imaging, they also sometimes give frustratingly low signal levels, according to Professor Loh. On the other hand, electrons interact very strongly with matter, providing good imaging contrast in electron microscopy images, but at the expense of sample damage.

Knowing the contrast mechanism in both cases is then tremendously helpful for interpreting what is being measured.

Professor Loh is extremely inspired by the ability to put affordable computing and efficient software packages into the hands of the wider community. Motivated by this cause, Professor Loh and his team is currently furthering their research in incorporating learning algorithms with this unique imaging technology.

They have combined elements of unsupervised clustering methods, with their own outlier detection routines, as well as modified Naive Bayes classifiers to work with the specific topology of the 3D rotation of the subject. The team is also looking into optimising the programmes for problems such as iterative phase retrieval.