The next generation laboratory

Technology in the laboratory has changed a great deal in a very short time much to the benefit of those working in the environment. Just a few short years ago, a piece of laboratory equipment like a microscope was simply a tool: An extremely sophisticated device capable of providing truly remarkable images, but not in itself capable of providing an answer. Only the scientist using it could do that.

This is no longer the case. As has been seen in countless other sectors from automotive to banking, retail to aerospace, the equipment we use in medical science has evolved beyond being just the picks and shovels—it can support us in unlocking solutions that weren’t possible before.

Since I left the world of research and joined Leica Microsystems in 2007, my focus has been on what I believe is now the most important part of any laboratory device: The software that sits behind it. This software can not only improve the quality of imaging, it can process volumes of information that would take a human researcher years to analyze and reveal information that might never otherwise be found.

Alongside natural language processing and generation, image processing is one of the most exciting areas in artificial intelligence (AI)—the full potential of which we are only beginning to realize. AI has the power to transform medical science: Removing some of the biggest research bottlenecks, rapidly advancing the speed at which we can analyze biological information, and expediting scientific discoveries in areas like developmental biology, cancer research, neuroscience, and immunology.

Quantitative shift

The problem scientists have always faced in microscopy is the ability to process large volumes of images quickly. It’s not quite Google scale of big data, but for a single research project we could be talking about hundreds of thousands of images or more—far too many for a human to evaluate efficiently and comprehensively. As a result, microscopy has always been a very descriptive discipline, without the quantitative rigor that can be more easily found in other areas of laboratory research.

AI is empowering us to automate this process and move toward a far more quantitative approach to image analysis. There are now intelligent systems that can process huge quantities of images and use deep learning techniques to self-improve as they go. The human still needs to provide ‘training’ data for the neural network to learn the correct mapping. But once trained, the algorithm can analyze huge quantities of image data and detect patterns that would not have been identified otherwise. While we don’t expect a move away from ‘traditional’ biology in the foreseeable future, elements such as human supervision may be made easier, as machines learn how to cluster information automatically or navigate a complex environment on their own.

The impact on scientific progress is going to be significant. Historically, to investigate a particular change in a cancerous cell, for example, you might need to look at 1,000 different proteins. That could potentially be 1,000 different PhD theses by 1,000 different people, with thousands of images viewed by each one. An AI could complete this analysis in less than a day. In addition, because it’s working across the piece and learning as it goes, it can identify interaction patterns that the others couldn’t possibly hope to have identified when looking at each protein in isolation. As a result, we can expect a dramatic productivity increase, mediated by AI, which will accelerate the pace of discovery by orders of magnitude.

Scientists driving change

Companies like Leica Microsystems are investing heavily in AI and automation technologies because we believe that this is going to be indispensable in the future. However, it’s the scientists and researchers that are driving this change by pushing existing microscopy technology to the very edge of what’s possible.

There is currently a major trend toward live samples, especially in areas like immunology and the study of infectious diseases. Model organisms enable you to see biological changes in real time—understanding how an organism reacts to a virus, or how multiple infections affect the organism differently when introduced simultaneously.

One of the big areas of AI development and investment has been in improving image quality for model organisms, tissue sections, and 3D cell cultures like organoids. Using techniques like computational clearing, it’s now possible to use camera-based fluorescence microscopes to capture images of thick samples without the out-of-focus blur that made this method previously unviable. Coupled with AI-driven image processing, this vastly increases the information gained from images that can be captured and interpreted in a short space of time.

In the past this has raised concerns about image validation, with some in the scientific community questioning whether images that have been ‘processed’ in this way can be interpreted quantitatively. But we’re starting to see this skepticism reducing as researchers begin to realize that slight imperfections introduced by the physics of the image acquisition itself can be reversed by AI. While the notion of software becoming an integral part of this acquisition process requires a good deal of rethinking, the upside is enormous. Being able to push microscopy over these entrenched boundaries leads to a paradigm shift away from the focus on the ‘raw image,’ and toward the microscope as a source of quantitative information.

Changing skill sets

It’s not just mindsets but skill sets that are set to experience a shift. More and more we’re starting to see those with skills you’d associate more with Google or Facebook—coding, engineering, data science—showing interest in medical science as the exciting possibilities that AI and automation bring to this sector make it increasingly attractive for those with a software engineering background.

Does this mean fewer opportunities for more traditional biologists? Actually, the opposite. The rise of AI will be liberating for scientists and researchers who often find themselves mired in tedious and time-consuming tasks in the laboratory. Technology is finally reaching the level of maturity where these tasks can be automated. Leica Microsystems has an automated research assistant in our

Figure 1. HeLa cell spheroid stained with Alexa Fluor 568 Phalloidin (Actin) and YOYO 1 iodide (Nucleus). Acquired with a standard wide field fluorescence microscope (left half) and an exposure of the same sample taken with a THUNDER imager (right half).

Personal AUtomated Lab Assistant (PAULA) system. Now we’re looking at how we can deploy this technology for microscopy, in a device that could look at an image of a sample on a regular basis and automate a response when there are changes—all without ongoing human intervention. It’s not about cutting out the experts. It’s about freeing up experts to do more value-adding tasks and empowering them to use AI to do what a human would never be able to do. Figure 1 and Figure 2 are two examples of images with computational clearing.

Figure 2. The LIGHTNING detection system for image information extraction ensures the clear and unobstructed view of the relevant information. LIGHTNING works with a decision mask (middle) that takes individual volume segments into account which is remarkable superior to a global processing. — Figure 2. The LIGHTNING detection system for image information extraction ensures the clear and
unobstructed view of the relevant information. LIGHTNING works with a decision mask (middle) that takes individual volume segments into account which is remarkable superior to a global processing.

Classification of adherent cell lines in fluorescence microscopy using a neural network classifier. This figure shows clusters of image features (i.e. patterns) detected by the network superimposed with ground-truth knowledge of the nature of cell lines (cell line names shown in legend). The network was trained using a subset of data from the Human Protein Atlas.

The age of AI

The Human Protein Atlas is a Swedish-based initiative aimed at mapping all human proteins in cells, tissues and organs. All the data in the knowledge resource is open access to allow anyone to pursue exploration of the human proteome. One of the finest examples of AI best practice in action is the work of Emma Lundberg’s SciLifeLab on the Human Protein Atlas. A decade’s worth of invaluable work, this project has specifically stained virtually every protein in vertebrate cells and imaged where they go with a microscope. Now, we have a colossal bank of images that we can begin to analyze with deep learning techniques to determine patterns in subcellular localization of proteins.

Leica Microsystems recently sponsored a competition through Kaggle¹ to create an algorithm to sort images into 28 classes, each showing different organelles of the cell. Data scientists comprised of 2,172 teams worldwide took on the challenge of predicting where a protein has gone in the cell with only an image as a cue. This task is particularly tricky as some proteins go to multiple places at the same time and some of those patterns are very rare. To succeed the data scientists had to search for independent data on the web to complement the competition data, use models pre-trained on millions of photographs, and fine-tune them to recognize microscopy images or to exploit techniques borrowed from facial recognition. Endeavors like this will help us to understand the role of proteins in health and disease on the level of individual cells.

This is where we as an industry need to be focused in this new era of technological advancement. By no means are we seeing the end to traditional biology; we are seeing biology about to be immeasurably enhanced by nascent but fast-developing techniques in AI and automation. Yes, there are still unknowns and we need continued investment in these areas to realize their full potential. But everyone needs to acknowledge that the age of AI is upon us—either we embrace it now, or we miss the chance to take advantage of a world of new possibilities.