‘Lifesaving’ X-Ray systems to work autonomously using Nvidia AI

Ultrasounds and X-Rays are the most common and widely-used diagnostic imaging systems, yet they are not readily available for nearly two-thirds of the global population. AI-enhanced robotic systems could change that, according to two partners on a new project.

The collaboration between AI hardware and software company Nvidia and GE Healthcare aims to develop autonomous X-Ray technologies and ultrasound applications.

“Building autonomy into systems like X-Ray and ultrasound requires medical imaging systems to understand and operate in the physical world. This enables the automation of complex workflows such as patient placement, image scanning and quality checking,” an Nvidia announcement said this week (18 March).

Health technology company GE Healthcare will use Nvidia’s new Isaac for Healthcare simulation platform. Including pre-trained models and physics-based simulations of sensors, anatomy and environments, the system is designed to accelerate research and development, enabling GE HealthCare to train and test autonomous capabilities in a virtual environment before deployment in the physical world.

“The healthcare industry is one of the most important applications of AI, as the demand for healthcare services far exceeds the supply,” said Kimberly Powell, vice-president of healthcare at Nvidia.

Autonomous imaging systems could “improve patient access and address the challenges of growing workloads and staffing shortages in healthcare”, added Roland Rott, president and CEO of imaging at GE HealthCare.

Other customers will use simulation environments on the Isaac platform, Nvidia said. These are designed to enable robotic systems to safely learn skills in a physically-accurate virtual environment for real-world situations, such as surgery, that would otherwise be impossible to replicate. The platform is designed for ‘multi-scale simulation’, ranging from microscopic structures to full hospital facilities. 

The system includes AI models fine-tuned for healthcare robotics that can “understand, act and see” using enhanced vision and language processing, Nvidia said. It can also work alongside Nvidia Holoscan, an edge computing platform, to power robotic decision making in the real world, in real time.

Developers can now access physics-based digital twins of medical environments, allowing them to import custom sensors, instruments and even anatomies to teach robots how to respond to various scenarios. “These virtual environments help close the gap between simulation and real-world implementation, and enable rapid digital prototyping,” the announcement said.