As a computer science student in the 1970s, Henry Fuchs tried to explain the benefits of affordable, personal computers to others at his school. Most were not convinced –– they saw no need for this technology in their day-to-day life.
“It was such a radical notion, even 10 years later I couldn’t convince professors in major universities that this was a good idea,” he says while throwing his hands up, still exasperated by the thought.
Decades later, the average person spends about 4 hours per day on their pocket-sized computer –– their smartphone. Fuchs doesn’t have time to relish in his correct prediction, though. He’s already looking to the future.
Fuchs’ research group focuses on computer graphics and virtual and augmented reality, with many of their project applications in the medical field. One of these is in laparoscopic surgery.
Also known as minimally invasive surgery, it is performed in a patient’s abdomen or pelvis using small incisions, a camera and rod-like laparoscope to transmit images, and specialized instruments. While patient recovery time is normally faster compared to an open surgery, there are challenges with this technique –– like training.
Timothy Farrell, a surgeon at UNC Health Care, says medical students first learn open surgery and then move on to laparoscopic surgery. Although the steps of both operations are essentially the same, the skills required vary greatly.
“They have to learn how to interact in this environment, which is no longer looking down through an opening in the patient’s abdominal wall, but rather looking at projected images on a screen while your hands are working below you,” he says. “It’s a whole different skillset. There’s a different learning curve.”
Fuchs hopes to ease this transition. For the past two years, his team has been developing an augmented-reality training tool for the procedure. With this technology, instead of looking at a TV, the trainee uses the augmented-reality headset display and laparoscopes to move objects from one side of a pegged board to the other. Through forced-feedback, the program will also be a way for students to fine-tune their techniques.
“It’s sort of like the expert surgeon will be putting their hands on the novice to guide them through how things should be done more precisely,” Fuchs says.
Their goal is that, one day, all laparoscopic surgeries will use augmented-reality displays instead of the current use of camera and monitors. Fuchs realizes that this technology is far off, though, and remains focused on perfecting the training tool.
“You have to do the reconstruction very precisely,” Fuchs says. “Even one, two millimeters that you’re off will keep you from grabbing the little pieces properly.”
Luckily for Fuchs and his students, they have a host of collaborators to get them through challenges like this. Besides working with the computer science department and doctors at the University of Arizona, Fuchs is also partnering with surgeons, like Farrell, here at UNC.
“By continuing to collaborate with the medical experts and the robotics experts, we not only learn from them on a regular basis, we get much more exciting and better results than otherwise,” Fuchs says. “The spirit of collaboration makes our work much more relevant.”