Developing the Case for Hyperspectral Imaging in Surgical Practice

Hyperspectral cameras hold enormous potential. They truly ‘make the invisible visible,’ illuminating an object and then capturing its reflected light in many narrow bands. It allows the cameras to reveal much more information about that object’s physical properties than can be discerned with the naked eye.

The technology comes with many use cases – from precision agriculture (think of weed control and pest surveillance) to industrial machine vision (enabling applications such as corrosion detection or quality inspection) and environmental monitoring (e.g., spotting oil spills or plastic waste).

In the medical domain, hyperspectral cameras can also make for a revolution. Imagine surgeons being able to make decisions based on real-time information about tissues’ chemical composition at a molecular level, ‘seeing’ the oxygenation of blood flowing through arteries and vessels, or distinguishing in vivo between healthy and anomalous tissue, such as tumor cells.

It is a glimpse at a future that is much closer than one might think. Yet, while hyperspectral sensors and cameras have evolved tremendously over the past years, several limitations continue to prevent the technology from straightforwardly being used in surgical practice. For one, devices are typically quite bulky and thus incompatible with an operating theatre’s already-crammed sterile field. And integrating them into hospitals’ stringent clinical workflows is no easy feat either.

A recent research project took these challenges to heart. It investigated whether hyperspectral imaging technology can support the in vivo detection of low-grade gliomas (a diverse group of slow-growing brain tumors) – an arduous task, even for well-trained and highly experienced surgeons.

Exploring the in vivo detection of low-grade gliomas using a hyperspectral camera

Low-grade gliomas are a type of intrinsic brain tumor, i.e., tumors that arise directly in the brain. They often develop in young, otherwise healthy patients. While typically considered benign in origin, studies have shown that these cancer cells can expand at a rate of four to five millimeters annually and come with the risk of malignant transformation. Their early surgical resection has thus become a much-favored treatment option – although in vivo detection of low-grade gliomas and retrieving their exact demarcations is notoriously difficult, even with the aid of surgical microscopes.

This is partly because low-grade gliomas do not lend themselves to being detected through commonly used fluorescence methods. Instead, a magnetic resonance imaging (MRI) or computed tomography (CT) scan needs to be performed before the tumor’s resection, even though this merely allows surgeons to identify a general region of interest (instead of the tumor’s exact boundaries).

In other words, giving surgeons the proper tools to detect intrinsic brain tumors in vivo would make for an important step forward. And hyperspectral imaging technology shows great potential to do just that – with two presentations at this year’s BiOS conference reporting on a significant breakthrough.

It is a breakthrough that was realized by mounting imec’s snapscan VNIR 150 hyperspectral camera on a standard surgical microscope. This proved a compact set-up, generating hyperspectral data that have the potential to help detect low-grade gliomas in vivo. As a next step, these data can be fed into a deep-learning neural network, providing valuable insights on healthy versus anomalous tissue.

The set-up’s secret recipe: a small form factor in combination with high accuracy

Thanks to its small form factor (10 x 7 x 6.5 cm, and weighing 645 g), and its compatibility with standard C-mount optics, imec’s snapscan VNIR camera could easily be mounted on the surgical microscope. It made for a compact set-up that, going forward, could more smoothly be incorporated into hospitals’ stringent clinical workflows, contrary to the bulky systems used in previous studies.

Secondly, the imec researchers focused on how to acquire relevant data in a (sterile) intraoperative environment – with minimal adaptations to both the system and the data pre-processing methods. To do so, they had to rethink the camera’s existing spatial and spectral calibration methods and interface the device directly with the surgical microscope’s optics and light source. 

Lighting was one of the other parameters they studied. To avoid excessive noise, the imec researchers adapted the spectral range of the snapscan camera to match that of the microscope’s internal light source. Finally, they also looked at the system’s integration time modalities to find the right balance between saturation and noise, all while optimizing the system’s imaging parameters. 

Moving on to video-rate hyperspectral imaging for real-time tumor classification  

A first important learning, the project has allowed imec researchers to specify which hyperspectral bands are critical when interfacing with high-end surgical microscopes for an in-vivo classification of low-grade gliomas. And secondly, the underlying algorithms have proven to detect the presence of low-grade gliomas on unseen data with 80 percent accuracy. This is a major step forward.   

Admittedly, the set-up’s intraoperative use is premature. Yet, so far, the approach has successfully been tested using a clinical dataset of six patients at Belgium’s Leuven University Hospital. As such, these are very promising results that lay the groundwork for moving to video-rate hyperspectral imaging – which would allow researchers to explore the real-time detection of low-grade gliomas in surgical practice.  

This study was conducted in collaboration with Carl Zeiss Meditec AG and Belgium’s Leuven University Hospital. The results were presented at SPIE BiOS 2023 in the following papers:    

• Integrating hyperspectral imaging in an existing intra-operative environment for detection of intrinsic brain tumors, R. Vandebriel (imec)  
   
• Intra-operative brain tumor detection with deep learning-optimized hyperspectral imaging, X. Zhang (Carl Zeiss Meditec AG) 

 This article was previously featured in Laser Focus World.