UCLH Heart Hospital teams with Glassworks to create the HeartWorks real motion heart modeling system.

In 2006, three clinicians at University College London Hospitals (UCLH) Heart Hospital, Drs. Sue Wright, Andrew Smith and Bruce Martin got together to share their frustration about the lack of a realistic model of the human heart that could be used to teach cardiac anatomy. They came up with the idea of creating a virtual heart, and quickly realized that if it were based on an anatomical 3D data set, such a model could be used to generate a simulated ultrasound image. The idea had significant implications for training, as it could also be used to simulate the experience of transesophageal echocardiography (TOE or TEE), an examination difficult to practice, as it relies on inserting an internal probe to capture images of a patient’s beating heart.

When these clinicians shared their idea with a team of creative artists at a UK animation/visual effects company called Glassworks, things began to take shape. The goals were to create an anatomically accurate, computer-generated heart model that could instantly produce an authentic ultrasound representation, be animated to beat in real time to show changes in the heart’s shape during the cardiac cycle, and enable the doctor to view slices of the scene to gain valuable diagnostic insights.

Glassworks Develops the Virtual Heart Model

The clinicians and Glassworks named the project HeartWorks and relied on the high-performance computing and visualization power of Nvidia Quadro professional graphics solutions in the development and use of this first-of-its-kind clinical training tool.

Nvidia has a stated committment to supporting technology innovation in medical imaging and life sciences. Its technology enables medical device providers to develop and commercialize solutions that meet the high-performance computing (HPC) and demanding visualization requirements of the healthcare community. Through its involvement in recent years there have been some marvelous advancements and some unusual partnerships come together to deliver those innovations. HeartWorks is a prime example.

A small team of artists and developers at Glassworks began collating a vast set of digital heart images, including attending open-heart surgery to experience the human heart beating live. Wright, Smith and Martin worked with leading surgeons, cardiac morphologists, ultrasonographers and other experts to consult with Glassworks as the system began to develop. To create the complex realistic heart model and real-time animations, the Glassworks artists and animators used animation software running on workstations equipped with Nvidia Quadro graphics processing units (GPUs). These professional graphics solutions provided the processing power necessary to render highquality images at 30 frames per second—keeping the animations realistic and smooth.

“Immersive technology, where users can interact with 3D images on a screen, has been limited historically by the state of graphic technology, and by budgets,” said Hector McLeod, founder of Glassworks. “Nvidia’s breakthrough with their massively parallel GPU technology has changed that. What the Nvidia GPU does within HeartWorks is load and display extremely complicated models, and render them at 30 frames a second, in a series of events that are themselves complex. In any given frame we’re tapping the GPU to examine the model, slice it, annotate the views and display the two visualizations—the model and the ultrasound—in less than 1/30 of a second.”

Glassworks software engineer David Llewellen added, “Nvidia Quadro easily handles the sheer quantity of data we’re pushing through it. The model has a high polygon count—250,000—and we’re generating two simulations, plus we have a whole system of anatomic labels. To get the model to look graphically superior we also have highly detailed textures, the majority using OpenGL and running the GL Shader Language. We need all of that processing power at once to make sure HeartWorks delivers a real-time experience.”

Glassworks and the UCLH Heart Hospital clinicians commercialized HeartWorks through Inventive Medical Ltd. in London, which integrates, installs and supports the system for hospitals, labs and universities. When it is delivered to clients, the HeartWorks product is a turnkey system that includes the HeartWorks software—the interactive, virtual heart model and the ultrasound simulation program; a high-performance workstation equipped with an Nvidia Quadro professional graphics card; a monitor/keyboard/mouse; and a probe and torso mannequin to enable a hands-on experience for teaching the TOE/ TEE procedure (Figure 1).

Figure 1: This module allows real-time simulated TTE imaging of the virtual heart us- ing a life size manikin torso. The torso has soft skin with accurate, palpable anatomical landmarks to aid positioning of the handheld ultrasound probe. The screen display allows the user to identify the position of the probe on the virtual chest as well as to see the orientation of the ultrasound plane.

 

HeartWorks in Action at Duke university

One of HeartWorks’ early adopters was Duke University, specifically, the Department of Anesthesiology, Division of Cardiothoracic Anesthesia and Critical Care Medicine. The Department bought the simulator early in 2009 to teach residents and fellows in their advanced Transesophagael Echocardiography program. First, second and third year residents use it primarily to learn basic echocardiographic views and anatomy, and fellows in the advanced program utilize it to examine more subtle features. “Simulation technology has enabled us to take a quantum leap forward in our teaching,” said Dr. Madhav Swaminathan, MD, FASE, FAHA, of Duke University School of Medicine’s Division of Cardiothoracic Anesthesia. “This particular system essentially simulates the beating heart clearly. To explain how an ultrasound image is formed and how it correlates to anatomical features is extremely difficult. When you’re changing the image plane with a probe it’s hard to understand what parts of the heart you are seeing on the screen—because the heart is three dimensional, and you’re using 3D on a 180 degree plane. A simulator makes it possible to see side by side not only how an ultrasound image is generated, but what the cuts mean in a controlled, relaxed environment where you don’t have to worry about interfering with a patient’s clinical  care or taking too much time. This virtual environment technology gives residents a jump start.”

Figure 2: Wireframe representation of the heart model. The model can also be viewed in slices or as a solid, 3D beating model.

Dr. Swaminathan further explained that the primary advantage of the simulation technology is the real-time interaction residents and fellows experience. “When you can see the heart beating and modify aspects of it, cut it, and manipulate it at will versus being forced to do it the way a teacher wants is a significant breakthrough in echo education.” (Figure 2) For a look at the stunning photoreal, animated beating-heart model, visit www.heartworks.me.uk.

Following its success with the TOE/ TEE application simulator, the UCLH Heart Hospital/Glassworks team began developing another HeartWorks device, a TTE, which simulates the experience of an external ultrasound examination. It utilizes the same virtual heart model as the original HeartWorks application, but the interaction via probe generates a slice plane that goes through the lungs and ribs, which are also visualized.

In the development of the new TTE application, Glassworks is investigating further leveraging Nvidia’s Quadro GPU technology by writing shader tools in the Nvidia Compute Unified Device Architecture (CUDA) programming language to give them an additional performance boost. “We’d love to make further optimizations that make things run even faster, and look even better,” said Glassworks’ Llewellyn. “We will make full use of whatever developments Nvidia brings along next.

 

What’s Next?

It’s those “next developments” that are always being designed for at Nvidia; to anticipate challenges and deliver solutions that clinicians and device manufacturers can apply to meet those challenges in new ways. Nvidia CUDA programming language enables dramatic increases in computing performance by harnessing the power of the GPU. The latest generations of the Quadro and Tesla GPUs, based on the Nvidia Fermi architecture, for workstation and server side applications, are designed to deliver performance that greatly accelerates any medical professional’s workflow with the potential to help improve patient outcomes.

Specifically, Nvidia GPUs and GPU computing with CUDA have the ability to not only perform compute-intensive image reconstruction for generating high-resolution and complex 3D images, but also the ability to process the image data for realtime clinical enhancements, annotations and even computer-aided diagnosis. We believe features like these are critical to the improvement of patient outcomes. HeartWorks streamlines the training process of preparing physicians and technicians to become proficient in reading and responding to the new paradigm of interactive medical imaging.

Today’s medical device vendors are offering solutions that acquire, compute, render, display and store larger amounts of information than ever before. This trend will continue well into the future. Positron Emission Tomography (PET), 4D ultrasound, 3D advanced applications and Hybrid Imaging, the combination of modalities like CT and PET, are but a few of the examples of innovation that have changed the way clinical exams are or will be performed. Mobility is also a reality for today’s healthcare professional. Information and applications no longer reside within the imaging department. The proliferation of various handheld devices along with webbased applications allow for the accessing of information from any location.

Keeping all of this in mind, the overall objective is to improve diagnostic accuracy that in turn helps improve patient outcomes without sacrificing efficiency—which is an absolute requirement given the demands on our healthcare system today.

 

Glassworks
London, UK.
+44 (0)207 434 1182

Heartworks
London, UK
+44 (0) 203 447 9360

NVIDIA
Santa Clara, CA
(408) 486-2000