CSE Professor, Lecturer Team with UC San Diego Health to Bring 3D Visualization to Abdomenal Surgery

Turning a fairly routine procedure into a bold experiment, CSE Professor Larry Smarr, director of Calit2, recently underwent a segmental colon resection (removing an eight-inch section of his large intestine) prefaced by the creation of dramatic, data-driven computer models of key organs in his abdomen. Both before and during the surgery, UC San Diego School of Medicine surgeon Dr. Sonia Ramamoorthy had access to 3D visualizations made possible in conjunction with Calit2's virtual reality team, including Jurgen Schulze, a lecturer in CSE who is a research scientist in Calit2's Qualcomm Institute. In an article published in today's This Week @ UC San Diego, "Visualizing the Future of Surgery", physicians and technologists talk about the operation and how this 'quantified surgery' translational pilot can impact the future of abdominal surgery. (Click here to read more on the new Quantified Surgery website, or click here to watch a four-minute video about the collaboration.)

What follows is more on the technologies that make quantified surgery possible, and the role played by QI and CSE personnel before, during and after the operation.

Larry Smarr began developing time series of a wide variety of measurements on his body, including blood and stool samples, in the mid-2000s. Initially focused on increasing exercise and modifying his nutrition for better health, he began noticing blood and stool biomarkers that indicated he was developing chronic inflammation -- at times hitting peaks far above what's considered normal. In summer 2011, when Smarr became the patient of UC San Diego’s Dr. William Sandborn, a national leader in clinical research on Inflammatory Bowel Disease (IBD), Smarr was surprised to find that one of his stool sample immune biomarkers had jumped to 125 times above the normal healthy upper limit. On reading the research literature, Smarr found that this biomarker was  a sensitive and specific indicator that the patient is suffering from active IBD. After a colonoscopy and CT abdominal imaging, Dr. Sandborn concluded that it was likely that Smarr was suffering from colonic Crohn’s disease, a subtype of IBD.

Sandborn followed up with an MRI of Smarr in January 2012, after which Smarr asked for the DICOM files of the MRI slices and gave them to Jurgen Schulze to convert the MRI slices to a 3D image of his abdomen. Schulze stacked the MRI DICOM images to create a 3D volumetric data set that could become transparent under interactive control (pictured above). Schulze then went one step further and had his team create an accurate physical model (left) of the diseased portion of his sigmoid colon by sending the coordinates of the 3D surface of the virtual sigmoid colon to a 3D printer. 

Fast forward to 2016. After continuing to track his health in minute detail, Smarr had plenty of evidence that his Crohn’s disease was worsening. Indeed, the wall thickness of his sigmoid colon had, over the course of eight years, become increasingly thickened.  In June 2016 Smarr traveled to Irvine, CA to have a millimeter-resolution full-body scan from Dr. Harvey Eisenberg, a pioneer of this technology.  Dr. Eisenberg also performed a virtual colonoscopy (see picture below) in which the colon is inflated with air and then imaged with the CT scan. Remarkably, it showed that in the inflamed portion of the sigmoid colon the walls had become so thick (seven times normal thickness) that the air could barely get through.  Dr. Eisenberg recommended a resection of the inflamed region.

On August 26, 2016 Smarr came to the conclusion that his worsening condition could lead to an emergency surgery while he was on travel, so he emailed his GI doctor William Sandborn that he believed the time had come for the laparoscopic removal of his sigmoid colon with its kink.

With this possibility in mind, Smarr met with Dr. Catherine Mohr, VP of Strategy for Intuitive Surgical, at the Exponential Medicine Conference in San Diego, who showed him their third generation da Vinci Xi surgical robot, which was optimized for colorectal surgery.  Knowing the UC San Diego had the da Vinci Xi, Smarr realized that the software of the da Vinci would allow for a 3D interactive model of his abdomen to be easily introduced into the da Vinci robot.

After talking with Smarr, Dr. Sandborn ordered a new MRI and colonoscopy and Dr. Ramamoorthy set a date, November 29, 2016, for the surgery. By early that month, an MRI and colonoscopy revealed the diameter of the channel ("the lumen") of the sigmoid colon had been reduced to 4mm from the normal 40mm.  “As long as I was going to go through surgery, I decided to turn it into a translational medicine pilot for what might be called ‘quantified surgery’,” observed Smarr.

Three weeks before the surgery, Smarr worked with the surgeon to host a pre-surgery planning meeting with about 16 physicians, engineers and scientists from surgery, radiology, GI, osteopathy, bioengineering, robotics, design and virtual reality. Smarr noted that physicians typically do not hold cross-disciplinary meetings for individual patient pre-surgical planning, but, he said, they were “quite excited to share their different insights.”

Then, in the two weeks leading up to the surgery, Smarr pulled together his own team at Calit2 to support the pre-surgery planning. Calit2 Research Scientist Jurgen Schulze took Smarr’s standard two-dimensional CAT and MRI scans and turned them into 3D volumetric computer models and organ segmentation visualizations. They then shared this 3D individualized "Google Map" of Smarr’s gut with the surgeon in pre-surgical planning.

“Essentially, our team applied software to the existing MRI digital inforrmation that was stored in the imaging section of my electronic medical record," noted Smarr. “By transforming the digital data into an intuitive, interactive 3D image, the surgeon was able to easily visualize  the specific characteristics of Smarr’s internal anatomy before the patient was on the table and opened up.”

Smarr and his medical team, including UC San Diego surgeon Sonia Ramamoorthy, discovered three key factors that would prove to be critical during the surgery: 1) the sigmoid colon had major ‘kinks’ at both ends; 2) the sigmoid was sitting on the bladder; and 3) the upper part of the large intestine had three turns at the splenic flexure and was likely attached to the spleen. 

“Dr. Ramamoorthy had never had this level of anatomical planning in her 20-year career and was so impressed, she drafted Jurgen Schulze into the operating room,” recalled Smarr. “She had him pipe our 3D anatomy into the da Vinci surgical robot, so she could see both the internal stereo laparoscopic camera view of me and the 3D anatomy as a guide.”  (Subsequent to the surgery, Dr. Ramamoorthy invited Smarr to give a talk to the doctors attending the Surgery Grand Rounds about making the leap. Click here to watch a video of Smarr's Grand Rounds presentation. Or click here to download his Powerpoint slides from the presentation.)

The da Vinci TilePro system permits viewing of the feed from the surgical camera on external monitors, which also permit viewing of additional external video sources. Once in the operating room, Schulze connected his laptop to the TilePro via HDMI and treated the da Vinci system as if the laptop were connected to a projector for a slide presentation. 

Surgery was scheduled for 7:30 a.m. on Nov. 29, and Smarr notes that the team was only able to test their combined visualization system the previous afternoon.

“UC San Diego Health Sciences was so excited about this translational medicine pilot that they put a full video crew into the operating room,” noted Smarr.

Post-surgery, Smarr was in the Jacobs Medical Center for four days and then at home for several weeks for rehabilitation. “Fortunately, I have had very little post-surgery pain, my new colon started up in two days and is working well, and I got back to a normal diet in a week,” explained Smarr. “Since the docs said walking was the best therapy, I started the evening after the surgery – dragging my IV pole and oxygen tank down the hospital corridors.”

Within two weeks, Smarr had ramped up to walking more than five miles per day. Having lost the diseased eight inches of his sigmoid colon, for the first time in 12 years, Smarr’s CRP dropped below 1 and his Lactoferrin value returned to normal.

"My greatest hope is that through my pilot experiment these consumer-driven visualization technologies will become a commonplace part of abdominal surgery in the near future," said patient Smarr. "We will do our best at UC San Diego to lead that change."

Calit2's Qualcomm Institute is now in discussion with UC San Diego Health and units on campus, including the QI Design Lab (directed by Don Norman), on forming a cross-disciplinary team to extend Smarr's pilot project to more patients.  As is characteristic of Calit2, this team is hoping to include private sector companies such as GE Healthcare, which is currently exploring with UC San Diego future-oriented imaging and intervention paradigms that assimilate new technology with human-centered design processes.