It’s been a frustrating afternoon for Andrew Rubin, a 49-year-old English professor whose right hand operates like a human Swiss army knife.
In one mode, his prosthetic device can pick something up. In another mode, it can turn a doorknob. But he can only perform one task at a time.
“If I want to change the grip,” Rubin says, “I have to use an app on my phone.”
Since his disabled hand was amputated a year ago, Rubin has become an early adopter of cutting-edge prosthetic devices. The process has turned him into a kind of bionic man, trying to perfect the symbiosis of human and machine.
Imagine drinking a glass of water or picking up a book with a tool on the end of your arm, rather than fingers that can respond to textures and surfaces. Now, imagine if that tool could, essentially, think. That’s the type of device Rubin is testing and training to use.
Its sophisticated software system relies on a form of artificial intelligence called pattern recognition. Ideally, it would allow Rubin’s hand to change grips automatically just by picking up signals from his forearm — signals that start in his brain.
But at this point, it doesn’t always work.
“I began to think of limbs as replaceable parts. My hand didn’t work, so I could get a better hand. But the hand is so much more complicated.”
Rubin’s journey to becoming a bionic man started 15 years ago, after his first year as a tenure-track professor at Georgetown University. That summer, he fell sick with something that felt like a bad flu. For months, doctors could find nothing wrong with him, until he collapsed with internal organ failure.
Rubin was eventually diagnosed with a bacterial infection, then developed complications from his treatment for a septic infection. After a partial recovery, he returned to work full-time, but the problems persisted. He left the university and required additional surgeries and treatments before deciding to have his lower leg amputated in 2017.
After the amputation, Rubin was fitted with a prosthesis. He felt more mobile, able to walk and exercise more.
Then came the decision to do the same with his hand, which had been paralyzed for nearly a decade because of the infection. Unlike a soldier returning from combat or accident victim emerging from a crash, Rubin had to arrive at a mental state where he would voluntarily undergo a second amputation.
“I began to think of limbs as replaceable parts,” Rubin says. “My hand didn’t work, so I could get a better hand. But the hand is so much more complicated. It’s the most complicated part of the body. It’s almost a work of art.”
Rubin is one of 158,000 people in the United States each year who undergo an amputation, usually for medical reasons. Many will decide to have their limbs replaced with prosthetic arms or legs that can cost between $15,000 for a simple hook-up to $150,000 for a sophisticated bionic arm and shoulder combination.
Rubin’s leg, custom-designed at MIT, contains a special battery-powered ankle that gives him a slight push on each step, making it easier for him to walk up hills. It would retail at $27,000, though his insurance covered the cost. Insurance also paid for his custom-designed hand with a list price of $36,000 — though earlier this year, he had to return the hand for several months until he got approval for its coverage.
Now, Rubin is using that hand to help software developers test their innovations. As materials, circuitry, and software have become smaller, lighter, and smarter, so too have the control systems that operate these prosthetics. The developers’ goal is to make them so small and intuitive that they integrate seamlessly into an amputee’s life.
Rubin is one of 10 patients learning to use the pattern recognition system at Infinite Biomedical Technologies, a Baltimore start-up that builds the brains of prosthetic arms and legs to make them work like regular ones. (Another firm, Coapt Technologies in Chicago, unveiled a similar system that’s being used by about 400 patients nationwide.)
Rubin’s training takes place inside a second-floor office near Baltimore’s Inner Harbor. Engineer Damini Agarawal attaches six electrodes to his upper forearm and plugs them into a black box. The box connects to a Surface tablet via wireless Bluetooth, and records nerve signals that Andrew generates in his forearm.
Those signals are sent to a microchip inside the prosthetic, and stored as a pattern or template, Agarawal explains. But on this day, because the new prosthetic test limb at Infinity doesn’t fit very well to Rubin’s stump, it’s not picking up the signals it needs to work properly.
“It’s not a perfect process,” Agarawal says. “The pattern of signals for ‘hand open’ and ‘hand closed’ are very close together. When he tries to perform one of these movements, the algorithm might be confused to which one it is.”
Still, Rubin sticks with it, as he has nearly every week for the past year, first with his doctors, then a prosthetist, now with software programmers. He is also working with engineers on two other experiments. One, performed by Infinite Biomedical Technologies with funding from the National Institutes for Health, uses RFID tags that are placed on objects in Rubin’s home. The RFID signal is scanned by software in his prosthetic arm, which then automatically changes the grip from, say, turning a doorknob to picking up a newspaper.
Meanwhile, a group of undergraduate students at nearby Johns Hopkins University is using Rubin in an experiment to connect his prosthetic to a mouse beside his laptop. The signal is picked up by nerves that used to connect to his fingers, and then is sent directly to the computer.
Being able to right-click and left-click a mouse is a huge deal for a professional writer like Rubin, who has continued his scholarship on 20th century philosophy, and recently finished editing a book of essays by noted literary critic Edward Said.
He likes being a techno-guinea pig — and acknowledges that he’s lucky. Many of these experiments are only first steps and aren’t available to the public. And without research funding, the cost would be overwhelming: No two amputees have the same kind of limb or nerve endings, and that means each prosthetic must be fitted individually.
Rubin, who has started using the pattern recognition software at home, says he’s looking forward to the day when he can use his new hand to take photos with his camera. He also wants to get back to the classroom and start lecturing again. It’s a long journey to turn a carbon-and-plastic tool into a dexterous hand that will make his life easier, but Rubin has made huge strides in the past few months.
“Walking my dog is a real pleasure,” he says. “I forget I have my new hand sometimes. My dog was running for a squirrel and pulled off my hand. I picked it up and screwed the hand back in.”