A new artificial intelligence tool that interprets medical images with unprecedented clarity does so in a way that could allow time-strapped clinicians to dedicate their attention to critical aspects of disease diagnosis and image interpretation.
The tool, called iStar (Inferring Super-Resolution Tissue Architecture), was developed by researchers at the Perelman School of Medicine at the University of Pennsylvania, who believe they can help clinicians diagnose and better treat cancers that might otherwise go undetected. The imaging technique provides both highly detailed views of individual cells and a broader look of the full spectrum of how people’s genes operate, which would allow doctors and researchers to see cancer cells that might otherwise have been virtually invisible. This tool can be used to determine whether safe margins were achieved through cancer surgeries and automatically provide annotation for microscopic images, paving the way for molecular disease diagnosis at that level.
A paper on the method, led by Daiwei “David” Zhang, PhD, a research associate, and Mingyao Li, PhD, a professor of Biostatistics and Digital Pathology, was published in Nature Biotechnology.
The development of iStar was taken on as part of the field of spatial transcriptomics, a relatively new field used to map gene activities within the space of tissues. Li and her colleagues adapted a machine learning tool called the Hierarchical Vision Transformer and trained it on standard tissue images. It begins by breaking down images into different stages, starting small and looking for fine details, then moving up and “grasping broader tissue patterns,” according to Li. A network guided by the AI system within iStar uses the information from the Hierarchical Vision Transformer to then absorb all of that information and apply it to predict gene activities, often at near-single-cell resolution.
To test the efficacy of the tool, Li and her colleagues evaluated iStar on many different types of cancer tissue, including breast, prostate, kidney, and colorectal cancers, mixed with healthy tissues. Within these tests, iStar was able to automatically detect tumor and cancer cells that were hard to identify just by eye. Clinicians in the future may be able to pick up and diagnose more hard-to-see or hard-to-identify cancers with iStar acting as a layer of support.
Perelman School of Medicine at the University of Pennsylvania release on Newswise
