Measuring physiological signals using contactless thermal infrared imaging

Wearable technology that can take various physiological measurements from the human body is well established. However, for long term use this technology can be obtrusive, it can give inaccurate readings, and it is not suitable for use by people with certain disabilities. PhD student Jitesh Joshi is exploring and improving a contactless way of measuring physiological signals that will help to solve these issues.

The most common way for people to self-track aspects of their health is currently through wearable devices. These devices can measure physiological metrics such as oxygen saturation, skin temperature, breathing rate and heart rate – which are all important for proactive health monitoring. But the limitations with wearable technologies – particularly in terms of being obtrusive in long term usage, as well as giving less reliable measurements during excessive movement – means alternative solutions must be found.

Researchers have shown the potential of thermal infrared imaging for tracking physiological signals in a contactless way. This method relies on thermal cameras that do not need to make contact with the skin and do not require ambient lighting to get readings. Compared to colour images, individuals are less identifiable when thermal imaging is used, leading to less privacy concerns too. However, there are computational challenges which need to be addressed before this method can be widely used.

Improving thermal infrared imaging

In response to this, Jitesh Joshi is focusing his PhD research on addressing the current challenges with thermal infrared imaging, by developing a computational framework. Joshi explained;

But before these devices can be used in such scenarios, the technology and reliability of the readings need to improve. One of the challenges with contactless thermal infrared imaging is that occlusions such as forehead hairs and eyeglasses create artifacts – which lead to inaccurate readings. Factors such as varying thermal ambient conditions and excessive movement can also make readings inaccurate.

Part of Joshi’s research spans the segmenting of facial regions and tracking the physiological signals coming from thermal infrared images acquired in a contactless way. By taking multiple frames of an image and segmenting the regions of each image, a computational framework can be established to better analyse what the images are depicting. In turn, this can lead to more accurate readings.

The AI-based computer vision framework that Joshi has developed in this context is showing improved accuracy in the identification facial regions in the presence of occlusions, ambient thermal conditions and low-resolution settings. This is a significant step forward in this area of technology, as this means it is becoming more robust in real world scenarios.

Next steps

Joshi has recently finished the first stage of his research, which is to gather a series of readings to feed into the algorithms. The next step is to apply the algorithms to video, to refine the effectiveness of the technology with regards to moving images. Once he is certain that the physiological signals the computational framework can extract are reliable, Joshi wants to create an application or device where it can be used in the real world.

“The question I’m ultimately looking at is, without contact, how do we reliably extract the physiological signals?” Joshi said. “This technology uses a deep learning architecture, a learning strategy and thermal imaging – which is actually very basic technology. This should make it highly usable and accessible when it is fully developed in the future. I also plan to make the AI-based algorithms publicly available, so that the technology can help all kinds of users.”

Funded by: Overseas Research Scholarship, Graduate Research Scholarship.