Robots, video games, and a radical new approach to treating stroke patients.
In late October, when the Apple TV was relaunched, Bandit’s Shark Showdown was among the first apps designed for the platform. The game stars a young dolphin with anime-huge eyes, who battles hammerhead sharks with bolts of ruby light. There is a thrilling realism to the undulance of the sea: each movement a player makes in its midnight-blue canyons unleashes a web of fluming consequences. Bandit’s tail is whiplash-fast, and the sharks’ shadows glide smoothly over rocks. Every shark, fish, and dolphin is rigged with an invisible skeleton, their cartoonish looks belied by the programming that drives them—coding deeply informed by the neurobiology of action. The game’s design seems suspiciously sophisticated when compared with that of apps like Candy Crush Soda Saga and Dude Perfect 2.
Bandit’s Shark Showdown’s creators, Omar Ahmad, Kat McNally, and Promit Roy, work for the Johns Hopkins School of Medicine, and made the game in conjunction with a neuroscientist and neurologist, John Krakauer, who is trying to radically change the way we approach stroke rehabilitation. Ahmad told me that their group has two ambitions: to create a successful commercial game and to build “artistic technologies to help heal John’s patients.” A sister version of the game is currently being played by stroke patients with impaired arms. Using a robotic sling, patients learn to sync the movements of their arms to the leaping, diving dolphin; that motoric empathy, Krakauer hopes, will keep patients engaged in the immersive world of the game for hours, contracting their real muscles to move the virtual dolphin.
Many scientists co-opt existing technologies, like the Nintendo Wii or the Microsoft Kinect, for research purposes. But the dolphin simulation was built in-house at Johns Hopkins, and has lived simultaneously in the commercial and the medical worlds since its inception. “We depend on user feedback to improve the game for John’s stroke patients,” Ahmad said. “This can’t work without an iterative loop between the market and the hospital.”
In December, 2010, Krakauer arrived at Johns Hopkins. His space, a few doors from the Moore Clinic, an early leader in the treatment of aids, had been set up in the traditional way—a wet lab, with sinks and ventilation hoods. The research done in neurology departments is, typically, benchwork: “test tubes, cells, and mice,” as one scientist described it. But Krakauer, who studies the brain mechanisms that control our arm movements, uses human subjects. “You can learn a lot about the brain without imaging it, lesioning it, or recording it,” Krakauer told me. His simple, non-invasive experiments are designed to produce new insights into how the brain learns to control the body. “We think of behavior as being the fundamental unit of study, not the brain’s circuitry. You need to study the former very carefully so that you can even begin to interpret the latter.”
Krakauer wanted to expand the scope of the lab, arguing that the study of the brain should be done in collaboration with people rarely found on a medical campus: “Pixar-grade” designers, engineers, computer programmers, and artists. Shortly after Krakauer arrived, he founded the Brain, Learning, Animation, Movement lab, or blam! That provocative acronym is true to the spirit of the lab, whose goal is to break down boundaries between the “ordinarily siloed worlds of art, science, and industry,” Krakauer told me. He believes in “propinquity,” the ricochet of bright minds in a constrained space. He wanted to create a kind of “neuro Bell Labs,” where different kinds of experts would unite around a shared interest in movement. Bell Labs is arguably the most successful research laboratory of all time; it has produced eight Nobel Prizes, and inventions ranging from radio astronomy to Unix and the laser. Like Bell, blam! would pioneer both biomedical technologies and commercial products. By developing a “self-philanthropizing ecosystem,” Krakauer believed, his lab could gain some degree of autonomy from traditionally conservative funding structures, like the National Institutes of Health.
The first problem that blam! has addressed as a team is stroke rehabilitation. Eight hundred thousand people in the U.S. have strokes each year; it is the No. 1 cause of long-term disability. Most cases result from clots that stop blood from flowing to part of the brain, causing tissue to die. “Picture someone standing on a hose, and the patch of grass it watered dying almost immediately,” Steve Zeiler, a neurologist and a colleague of Krakauer’s, told me. Survivors generally suffer from hemiparesis, weakness on one side of the body. We are getting better at keeping people alive, but this means that millions of Americans are now living for years in what’s called “the chronic state” of stroke: their recovery has plateaued, their insurance has often stopped covering therapy, and they are left with a moderate to severe disability.
In 2010, Krakauer received a grant from the James S. McDonnell Foundation to conduct a series of studies exploring how patients recover in the first year after a stroke. He was already well established in the worlds of motor-control and stroke research. He had discovered that a patient’s recovery was closely linked to the degree of initial impairment, a “proportional recovery rule” that had a frightening implication: if you could use early measures of impairment to make accurate predictions about a patient’s recovery three months later, what did that say about conventional physical therapy? “It doesn’t reverse the impairment,” Krakauer said.
Nick Ward, a British stroke and neurorehabilitation specialist who also works on paretic arms, told me that the current model of rehabilitative therapy for the arm is “nihilistic.” A patient lucky enough to have good insurance typically receives an hour each per day of physical, occupational, and speech therapy in the weeks following a stroke. “The movement training we are delivering is occurring at such low doses that it has no discernible impact on impairment,” Krakauer told me. “The message to patients has been: ‘Listen, your arm is really bad, your arm isn’t going to get better, we’re not going to focus on your arm,’ ” Ward said. “It’s become accepted wisdom that the arm doesn’t do well. So why bother?”
Krakauer and his team are now engaged in a clinical trial that will test a new way of delivering rehabilitation, using robotics and the video game made by Ahmad, Roy, and McNally, who make up an “arts and engineering” group within the Department of Neurology. Krakauer hopes to significantly reduce patients’ impairment, and to demonstrate that the collaborative model of blam! is “the way to go” for the future study and treatment of brain disease.
Reza Shadmehr, a Johns Hopkins colleague and a leader in the field of human motor-control research, told me, “He’s trying to apply things that we have developed in basic science to actually help patients. And I know that’s what you’re supposed to do, but, by God, there are very few people who really do it.”
“You bank on your reputation, in the more conventional sense, to be allowed to take these risks,” Krakauer said. “I’m cashing in my chits to do something wild.”
In 1924, Charles Sherrington, one of the founders of modern neuroscience, said, “To move things is all that mankind can do; for such the sole executant is muscle, whether in whispering a syllable or in felling a forest.” For Sherrington, a human being was a human doing.
Yet the body often seems to go about its business without us. As a result, we may be tempted to underrate the “intelligence” of the motor system. There is a deep-seated tendency in our culture, Krakauer says, to dichotomize brains and brawn, cognition and movement. But he points out that even a movement as simple as reaching for a coffee cup requires an incredibly sophisticated set of computations. “Movement is the result of decisions, and the decisions you make are reflected in movements,” Krakauer told me.
Motor skills, like Stephen Curry’s jump shot, require the acquisition and manipulation of knowledge, just like those activities we deem to be headier pursuits, such as chess and astrophysics. “Working with one’s hands is working with one’s mind,” Krakauer said, but the distinction between skill and knowledge is an ancient bias that goes back to the Greeks, for whom techne, skill, was distinct from episteme, knowledge or science.
One afternoon, a team from National Geographic came to the lab to interview Krakauer for a special series on innovators. I found myself eavesdropping on the reporters for whom he was simulating being Dr. Krakauer. At one point, the cameraman asked him, rephrasing Edison’s inspiration-perspiration binary, “Would you say that creative work is ten per cent mental and ninety per cent physical?”
“Actually,” Krakauer said, “the physical-mental distinction is something we’re trying to break down here.”
Krakauer is forty-nine, with soft, prairie-dog hair and keen eyes framed by boxy blue glasses. A serious tennis player in high school, he still plays squash competitively. He has one farsighted eye and one nearsighted eye, a detail that I’ll resist making too much of here. Even close friends say that Krakauer’s high standards can sometimes be maddening. “You are correct,” Steve Zeiler texted him late one night. “And exhausting.”
Although Krakauer went to Cambridge and speaks with a crisp British accent, he spent most of his early life in a small fishing village in southern Portugal. His father, a Jew, fled Germany in 1938, before joining the U.S. Air Force; his mother, who is British, was a teacher. After they separated, she moved with her children to Albufeira. When Krakauer was seven years old, his to-do list had one item: Bring pencil to Portugal. His favorite toy was a tiny gray hedgehog, which he believed was omniscient and controlled the world.
Edward O. Wilson has said, “It’s not a good idea to ask a scientist what he has read as literature.” But the library in Krakauer’s minimalist town house is extensive. His favorite book is “Gemini,” by Michel Tournier, a grotesque and hilarious tale of exiled twins who travel the globe in pursuit of their lost and future selves.
“John and I have always been dissatisfied with the options that presented themselves to us,” his younger brother, David, the president of the Santa Fe Institute, told me. “It has to be a this or a that, mutually exclusive choices—you’re a scientist or you’re an artist or you’re a historian.” Some of John’s wilder affiliations make him sound a little like a nerdier version of the Most Interesting Man in the World, from the Dos Equis ads. Once, I called him to discover that he was consulting with nasa on the projected Mars mission; in summers, he holds a post at a Portuguese biomedical-research facility called the Champalimaud Centre for the Unknown.
Krakauer got his medical degree at Columbia. At a lecture in 1996, the last year of his neurology residency, he asked a question about the pet scans of stroke patients that attracted the attention of Claude Ghez, a prominent figure in motor-control research. “Before I met Claude, I thought neuroscience was test tubes and cells,” Krakauer said. “Claude taught me that simple behavioral experiments could reveal profound truths about how the brain is organized.” Ghez asked Krakauer to join his lab. As Ghez told me, “John had a gift for identifying the question worth studying.”
We’ve all occasionally been frustrated by the gulf between what the imagination can choreograph and what the body can accomplish. But for stroke survivors this chasm can be permanent. Four years after a stroke, eighty per cent of patients still report impairment so severe that they have difficulty grooming, bathing, cooking, and driving.
Fifty years ago, nobody thought that the regrowth of neuronal connections after acute brain injury was possible. Now there is a widespread consensus that these connections are in constant flux.
Stroke-induced injury to the brain may have a silver lining, neurologically speaking. The tissue death that results from stroke appears to trigger a self-repair program in the brain. For between one and three months, the brain enters a growth phase of molecular, physiological, and structural change that in some ways resembles the brain environment of infancy and early childhood. The brain becomes, as one researcher told me, “exquisitely sensitive to our behavior.” What follows is a sort of “G.P.S. recalculating” period. Networks of brain cells begin to reroute around the stroke lesion, and neurons adjacent to the lesion start to take over some of the dead cells’ functions. S. Thomas Carmichael, a neuroscientist and neurologist at U.C.L.A., compared the period of plasticity to the explosion of seedlings after a forest fire: it’s a fecund time, but those shoots are tender, vulnerable, easily damaged. He cautioned that it’s essential to harness that growth. “You wouldn’t turn this growth phase on and plunk somebody in front of the television to binge-watch ‘Modern Family,’ ” he joked.
But, for many patients, that is essentially what happens. A 2004 University of Melbourne study, titled “Inactive and Alone,” showed that, in the early weeks of acute-stroke care, most patients spend fifty-three per cent of their time in their hospital beds. According to a later study, stroke patients who receive physical therapy for their paretic arm make, on average, thirty-two reaches per session. When neuroscientists perform studies on post-stroke mice, rats, and monkeys, the animals are required to make as many as four hundred to five hundred reaches per session. “Around thirty reaches per rehab session is having no impact on impairment,” Krakauer said. “We are providing physical therapy at homeopathic doses.”
Another problem, Krakauer said, is that patients are being prematurely made to learn compensatory strategies. They lean heavily on their good side to get out of bed, to get to the toilet, to wash and feed themselves. As one neurologist described it, learning such strategies can mean “the difference between having someone wipe your butt and wiping your own butt.” But Krakauer worries that the accommodations that make a patient more independent in the short term actually “stamp in suboptimal strategies.” True recovery, for Krakauer, would mean that a patient was able to move her paretic arm as she did before the stroke.
“We knew that spontaneous biological recovery happens early,” Krakauer told me. “What could one do to maximally take advantage of this unique period?” Everyone seems to agree that something special is happening in the brain during the first three months following a stroke; this is when most patients make their greatest gains. Nick Ward, the stroke specialist, told me, “John is certainly responsible for bringing this new agenda into focus, this question: What is it about this early phase which is really important—is this where to target our therapies?”
However, not all the doctors and therapists I spoke with think that beginning intensive movement therapy in the early weeks following a stroke is advisable, or even plausible. Carmichael suggested that it would be difficult to get stroke patients to stick to such a demanding regimen. He said, “You take a seventy-five-year-old who wasn’t very active, and you’re asking him to be more active than he was before the stroke.”
Krakauer agrees that what he’s proposing will require immense effort on the part of patients, as well as a restructuring of the entire delivery system for stroke rehabilitation. But a 2009 study that translated animal doses of therapy into human proportions provides hope that people can complete such a regimen. And Krakauer has a substantial ally in his quest to rehabilitate the arm: an “antigravity” robot.
The Hocoma ArmeoPower is a robotic arm that brings to mind the love child of a large dental chair and the Nintendo Power Glove. The patient’s arm is strapped into a motorized robotic sling, which can assist with movements or, in the first few days after a stroke, when patients might not be able to move their arms independently, even take over.
Rehabilitative video games can be used in conjunction with the Hocoma; some come preloaded. But most of them are focussed on functional, real-world tasks. The objective of one game developed for use with the robot is “grating carrots.” “Abysmal games,” Adrian Haith, a neuroscientist and blam!’s co-director, said, laughing. “The cleaning-the-stovetop game, the picking-the-apples-in-the-supermarket game.”
For Krakauer, the game had to be fun, and it had to be beautiful. Motivation and reward, he said, play a huge role in how we learn movement. “It’s not sufficient to say, ‘Take this, it’s a medicine,’ ” Krakauer said. “Physical therapy is boring and difficult and uncomfortable, and I planned to ask my patients to spend two hours a day working hard in this virtual world.” And why should we expect a poorly designed game to be an effective therapy? “It’s like saying to somebody, ‘When you are sick, you have to settle for black-and-white TV.’ ”
Krakauer needed a paradox, a “non-task-based task.” He wanted to encourage “childlike exploration” with the arm, in the workspace of daily life—the space around the torso, where we make most of our arm movements. Something analogous to what babies do when they’re learning how to speak. If conventional therapy was like repeating a single conjugated sentence, Krakauer wanted his patients to “babble,” trying out countless varieties of movement.
Since 2008, Omar Ahmad had been visiting the dolphins at the National Aquarium, in Baltimore. He often brought his son. The dolphins swam upside down, shovelling bubbles at Omar’s face with the muscular petals of their flukes. “I knew I was going to make a dolphin simulation,” Ahmad said. “I was fascinated by their locomotion.” Ahmad, who grew up in Eugene, Oregon, did his Ph.D. at Johns Hopkins. Part of his graduate work involved making mathematical models to predict hip fractures. “For hours, I watched people walk,” he said. “I got really good at studying human locomotion.” A former instructor of his introduced him to Krakauer, who was looking for a game designer. Although Krakauer wasn’t thinking dolphins quite yet, he had a vision of “something tumbling through indigo.”
Krakauer recalls taking a trip, in 2011, with Ahmad and Promit Roy, whom Ahmad met when they were both students at Johns Hopkins. “We went to San Francisco together, to the first Neuroscience and Gaming Meeting—that was our honeymoon,” Krakauer said. “And we just sat there in contempt of it. We thought the games were lame.”
Krakauer’s requirements for his stroke game meshed with Ahmad and Roy’s goals for a commercial game, and with their philosophy of design. Since 2008, Ahmad had been working on games that included animal locomotion; one of them, Aves, had been featured in the Apple store and earned him a fan letter from Steve Jobs. Now he wanted to make a “physics-based” game, with highly realistic movement. Roy recalls, “We thought that the way things moved in games was all wrong; we thought we could make it better.” They joined blam!, and a few months later brought on board Kat McNally, a twenty-five-year-old graphic artist and a graduate of the Maryland Institute College of Art, whose vibrant drawings and comics they admired. They form a subgroup called the Kata Project. (In Japanese, kata means “form.” ) Though they’re full-time employees of the Johns Hopkins medical school, the three also own an indie gaming company called Max and Haley.
One day, a staffer at the National Aquarium noticed that Ahmad was glued to the dolphin tanks; she suggested that he read Diana Reiss’s book “The Dolphin in the Mirror.” Reiss is a leading dolphin researcher at Hunter College and an advocate for the protection of dolphins. A friend of Krakauer’s, she introduced the team to her close friend, Sue Hunter, the head of animal programs at the National Aquarium.
Together, the group worked on the dolphin—“No, no,” Hunter would tell McNally, “the eyes lack a certain spark.” Or, “Yes, that dive posture looks right.” Last September, Hunter died of a brain tumor. Before her final surgery, the team took the finished game to show her. “Sue really loves it,” Ahmad said, when I visited the National Aquarium with him and McNally. Two months later, they were still mixing up verb tenses when they spoke about her.
The Pit is a spandrel in the center of the aquarium’s dolphin tanks, to which Kata has special access. McNally and I climbed down a ladder into a chamber the width of a closet, fronted by three windows that looked into the tanks. McNally told me that she’d spent thousands of hours in the Pit, sketching and observing. Far above us, light streamed through a domed roof, beyond which lay the Baltimore harbor and skyline. Fluctuating yellow beams of light, known as caustics, reached thinly toward our faces. Roy told me later that he’s been updating the code for the game’s ocean to include them. McNally stood beside me, breathing evenly. “Look,” she said, grabbing my arm. A dolphin had levitated, her gray face smiling like that of a beatific astronaut. In “The Dolphin in the Mirror,” Reiss describes these playful cetaceans as “minds in the water.” Playing requires a certain kind of intelligence, she points out; physical play is a way of learning what you can do in your environment.
Watching the animal’s fusiform body, it was hard to imagine a creature better suited to its environment. To climb out of the Pit, Kat and I pulled ourselves up using several cold red rails, reaching with our arms and legs, cinder-blocking our muscles against gravity. Below us, the dolphins flew around the teal space like large, strange birds.
Next, we visited an open saltwater pool. Ahmad told me that the team was adding some new marine characters to the game. I was peering down at their inspiration. Bright creatures slid thickly over one another. A ray looked like a steamrolled moon. Bluefish schooled around with yellow pouts, as if regretting their choice of lipstick.
Ahmad pointed to a blacktip reef shark that was making a labored U-turn. “You know how a stupid kid turns his head to answer but a smart kid will roll his eyes over to you without turning his head from the TV screen?” he asked. He pointed out that the shark’s flat eyes were aligned with its trajectory; a dolphin, however, has the ability to “decouple” attention from motion. In the video game, Bandit’s eye movements sometimes direct a player to an imminent threat, or roll downward to suggest where the dolphin wishes to move.
Later that afternoon, at the blam! lab, McNally had a picture of a calligraphic slash tacked up to her cubicle: it was “the first Bandit,” an impressionistic version of the cartoon dolphin now floating on Roy’s computer screen.
“Take the skin off for her,” Ahmad said.
They turned off the lights and disrobed Bandit. I was now staring at a skeleton, rotating gently in the ocean. What I saw was naked blocks of undulating color. And yet my brain interpreted the movement as a rippling spine. Albert Michotte, a Belgian experimental psychologist, demonstrated that if lines of light move in particular ways onscreen we have an innate tendency to view this activity as “living” movement. Just watching that shiver in the water, I knew it was alive.
“The way we did the animation is that every creature is a simulated biological entity,” Ahmad said. The team wanted to start with the warmth of hand-drawn animation and then apply the rigor of simulation. Ahmad said that he sees his work as part of a “forgotten lineage,” the old “wobble and stretch” of Disney cartoons. He calls this method the “road not taken” of modern gaming animation: hand-drawn characters, moving in springy, exaggerated ways. Mickey’s and Pluto’s shaky, hyperbolic reactions to gravity actually serve to enhance the cartoons’ realism, Ahmad said, by making invisible forces present to the viewer. “You’re watching physics in action,” he told me. “Muscles, friction.”
Today’s game animation, Ahmad said, feels “more like a stitched-together flip book. That’s because it doesn’t simulate the physics. You hit a button on the controller and it unspools a pre-recorded program; all the action is determined in advance and programmed into the game.” Roy showed me the game engine, which he had coded from scratch: files scrolled by my eyes, with names like “tail stun” and “awesome bubbles” and “even more awesome bubbles.”
Kata’s goal was to make an animal simulation realistic enough to activate what Ahmad calls the “motoric connection,” the visceral link between the patient’s moving body and the simulated dolphin; when Bandit leaps, you feel a tingling in your spine. “I created a field,” Ahmad said, with a fierce shrug, as if challenging me to disbelieve him. “The field of interactive animation.” (“Omar’s style is ‘burn the world down,’ ” Roy told me once, in his characteristic deadpan; if Ahmad’s style is a fiery idealism, Roy is the Kata straight man.)
The commercial game has been nominated for several honors for technical innovations by Pocket Gamer and AiGameDev.com. Stefan Schumacher, a thirty-two-year-old Pixar animator who is working on the sequel to “Finding Nemo,” visited the blam! lab in the fall of 2014 and came away impressed by what Kata had accomplished. “There are only three of them,” he marvelled. Top productions, like Grand Theft Auto and the Sims series, employ teams of hundreds. “Simulation is done entirely by a computer,” Schumacher said. “It’s very good at replicating physical behavior. What simulation is not very good at is making design choices. I think these guys are very smart about bridging these worlds.”
For months, the team struggled with the problem of how to give a player control over the simulation while still imbuing the dolphin with the illusion of independent life. At first, they considered splitting the player and the dolphin into trainer and animal. But that didn’t sit right; they didn’t want there to be a disconnect that made it feel “like Bandit was a toy you were tugging around on a leash,” McNally said.
One day, the team watched video footage of Omar waving his arm, with slow deliberation, at a dolphin in the aquarium; behind the glass, the dolphin mirrored him. The group was witnessing a moment of merger between two independent actors. The player, the team decided, would have to “become” the dolphin, adjusting his own movements to match the animal’s.
“Dolphins are like scientists,” Diana Reiss says. “They play to test the contingencies of their world.” The game the Kata team designed aims to productively disorient patients by plunging them into an ocean and asking them to learn how to swim the dolphin around. “There’s no right and wrong when you’re playing as a dolphin,” Krakauer said. “You’re learning the ABCs again—the building blocks of action. You’re not thinking about your arm’s limitations. You’re learning to control a dolphin. In the process, you’re going to experiment with many movements you’d never try in conventional therapy.”
Last November, I went back to Johns Hopkins to witness preparations for the trial. Seventy-two patients will be recruited within five weeks of their strokes; patients will also be tested at Columbia University and at a clinic in Zurich. Twice a day for three weeks, they’ll play the rehabilitative version of the dolphin game with the Hocoma robot for an hour; a control group will receive conventional occupational therapy.
Recruitment hadn’t yet begun, but several chronic patients had volunteered to help set the trial protocols; one afternoon, I slid into a hushed crowd of spectators in the main room of the blam! lab, on the second floor of Johns Hopkins’s Carnegie building. Roy was adjusting the Hocoma chair; on a large L.C.D. screen, Bandit, who is modelled on the Atlantic bottlenose, grinned out at us, his tongue humped and glowing like a radioactive scoop of strawberry ice cream. G., the volunteer, was a fifty-three-year-old man wearing a Puma baseball cap and a relaxed expression; he laughed out loud when he saw the dolphin. “Aw, man, I’m in awe you guys put this together,” he said. “Before I had my stroke, I’d play Wii for hours and hours.”
Krakauer was in clinician mode, conferring with the physical therapist and making adjustments to G.’s arm in the sling. His tone was calm and solicitous, but some of his questions were unusual: “Is that comfortable? Are you ready to get those mackerel?” G. oriented himself, moving his impaired arm around in the robotic sling.
The game’s settings are not easy. Strapped into the Hocoma chair, I had played it earlier in the week, and found it less like a video game than like the first ten minutes in an unfamiliar rental car. Dedicated physical therapists, Krakauer said, will tailor the game to each patient’s needs and abilities. For a while, G. simply explored the ocean, lifting and lowering his arm in the sling. Gradually, he learned how to make the dolphin jump, dive, and roll—the arm itself was the controller, with a motion-capture camera keyed to Bandit’s movements.
“It’s like floating in space,” G. later told me. “It’s not like lifting barbells or anything.” Then he got serious. He developed a strategy, waiting at the surface and then surprising the orange mackerel. Bandit, with a remorseless pink grin right out of “American Psycho,” ate the fish with bone-crunching gusto.
G. was able to play the game for almost forty minutes. “I was so wrapped up in the game, I wasn’t noticing that chair,” he said. Before his stroke, G. provided job training to people with special needs. Now he is unemployed and living on disability. Several months ago, his insurance stopped covering physical rehabilitation, despite his eagerness to continue.
G.’s eyes began to blink rapidly. “I am tearing up,” he apologized. For a long time, he said, he’d felt as though he’d “fallen through the cracks.” His arm had come to feel like a fleshy metonym for the rest of him, for the way the health-care system regarded him—as something broken, swept out of sight. “I would say that, for us, they just want you to cope with what you have left.”
In “Gemini,” the Tournier book, there is a passage in which the surviving brother compares his amputated limb to his dead twin. He describes his longing as a ghostly reaching through space. We often conceive of reminiscence as a backward movement through time, but everyone knows that grief can be disorientingly nonlinear, extending radially through the past and the future. The loss of motor function, according to many of the stroke patients with whom I spoke, is akin to a kind of bereavement.
In March, Krakauer flew with Roy to Zurich to set up a wing of the trial there. Patients with chronic stroke who tested the protocols loved the game, and the Kata team found this deeply gratifying. “In these cases, outside of the sensitive window we’re likely not reversing the deficit,” Krakauer said. But patient feedback at Johns Hopkins and in Zurich suggests that the game may provide an unexpected psychological benefit—a kind of “motor psychotherapy.” With the robot’s assistance, patients could use their semi-paralyzed arm to achieve something again, in the blue world of the game.
The game’s commercial launch has been a more equivocal experience for Roy, Ahmad, and McNally. Shark Eaters: Rise of the Dolphins, the precursor to Bandit’s Shark Showdown, débuted in the Apple store in October, 2014, the week that Krakauer’s trial was approved by the Johns Hopkins review board. With no marketing dollars, Max and Haley was hoping for reviews from industry publications such as Kotaku, and for a coveted spot on Apple’s featured apps chart. But, despite some positive reviews, the Apple feature didn’t happen, and the media attention the game received has focussed almost exclusively on its potential as stroke therapy. “Nobody knows about Max and Haley,” Ahmad said. Many people both within and without Johns Hopkins’s walls assumed that Krakauer had created the game.
McNally is the artistic director of the Kata Project, but there was no model for such a position in the School of Medicine. The administration initially tried to reclassify her as a medical-illustration intern. McNally has reddish-brown bangs that frame a round, wholesome face; she told me that people outside of blam! tended to view her as a sort of plucky graduate student. Her voice tightened as she recounted some recent interactions with journalists, potential donors, and certain people in the university’s administration. She’d felt invisible to them. “It’s my company, too,” she said. “It’s our game. And what we’ve done isn’t possible without all of us.”
Krakauer said, “Because I started this in my lab, and because there’s so much neuroscience and clinical interest, I get a little bit too much of the attention, and I don’t want that.” A sort of irony seemed to be at play here, he said: the tendency to wrongly dichotomize knowledge and skill, “intellectual” work versus “hands-on” labor, seemed to be getting recapitulated at an institutional level.
For artists like McNally, and for systems engineers and computer programmers like Roy and Ahmad, there are well-established career paths. Roy, who previously worked for Microsoft and for corporate gaming companies, told me, “I was all set to leave for industry until this happened. I can go out West—I can make a hundred starting, or I can make thirty-five here? That doesn’t make sense.” McNally said, “All of us hated this idea that you have to choose between academia and industry—that it’s one path or the other and they don’t ever cross. You do good, or you make money. We really wanted to do both things.”
Krakauer is anxious about his ability to retain the people who make up the Kata Project. “I came to Hopkins to set up a center,” he said. “This work cannot be done based on single investigators getting N.I.H. grants.” He feels that a group like Kata is essential to blam!’s work, and says they have the potential to become a university-wide resource: to “Pixar-up the health-care and scientific space.” But when he arrived there was no model in place to sustain such a group at Johns Hopkins. “We don’t know how to hire them, we don’t know how to pay them, and we don’t know how to acknowledge them,” he said.
Last May, a Silicon Valley company tried to recruit Ahmad from the medical school. I asked him what it would take to keep the Kata team together at Johns Hopkins. He jokingly sent me a link to a clip from “Jerry Maguire”: Tom Cruise yelling, “Show me the money.” In September, Ahmad was promoted to director of innovative biomedical engineering, a position created for him by Justin McArthur, the director of the Neurology Department at Johns Hopkins. McArthur told me that the School of Medicine is also clearing unused labs to provide blam! with room to grow.
Max and Haley has updated the game four times. While the cetacean locomotion grows increasingly realistic, other features of the game have become more fantastical, and slightly more violent. You can stun sharks with a “tail bolt,” and flood the ocean with fireworks that look like exploding pomegranates. Sales have been modest, but Roy told me that the game is developing something of a cult following. Ed Catmull, a co-founder of Pixar, stopped by the blam! lab during a visit to Johns Hopkins in May. Ahmad, describing the visit to me, was the happiest I’d ever heard him—Catmull is a hero of his, and the sophisticated whimsy of Pixar’s animation has been a huge influence on the Kata Project. Catmull told me in an e-mail that he loved the group’s energy, and its use of gaming machines. He wrote, “It is a great example of both original research and joining together the ongoing development of two different technologies. It will also teach us something about learning and brain development that will go way beyond helping stroke victims.”
Ahmad and Krakauer both credited the game’s high quality to its hybrid model of development. By designing the commercial game “to compete with the best games on the market,” Krakauer said, the Kata team was able to create what he feels is the best kind of game for neurorehabilitation. Ahmad said, “We had a choice: we could have made much simpler things. John would not let us do that. He made me do the hardest possible thing, the magic that is difficult.”
Perhaps one day the blam! lab will be able to “self-fund on the order of hundreds of millions,” as its Web site ambitiously proposes; as of this moment, it is still dependent on the traditional funding ecosystem. Susan Fitzpatrick, of the James S. McDonnell Foundation, which has funded the stroke-rehabilitation trial, said that she was pleased with its progress: “Regardless of what the outcome is, it’s going to tell us something important—in this case, when is the optimal time for delivery of rehabilitation? Success, as we define it, will be an answer to the question.”
She added, “Of course, we are hoping for a positive result.”
It will be several years before the findings are published. In the meantime, Krakauer and the blam! lab are aggressively pursuing various targets. The last time I visited the lab, they were ripping out more sinks to create a machine shop. The next steps, Krakauer said, will be the 3-D printing of assistive devices to help with the rehabilitation of hands and legs.
Krakauer and I had our final dinner together at a quiet Baltimore restaurant near his home. Diners’ faces bloomed and faded in the wavy light produced by many tiny candles. Krakauer seemed a bit beleaguered. There were many logistical challenges to recruiting patients during the acute period, he said. They were often very sick; they were geographically dispersed. Earlier that morning, he’d examined a friend’s mother in his home; the rest of the day had been spent setting up the trial protocols. His arms, which were usually ottery when he spoke, lay rigid on the table.
Then Krakauer’s vision began to toggle, his focus shifting from the myopia of daily administrative hurdles to the sprawling darkness at the frontiers of his field. He began telling me about upcoming blam! experiments investigating the universal laws of the brain, the “biological invariances” that underwrite all human movement. Now his arms were prowling the air, as if assisting with the invisible manufacture of meaning. Krakauer compared blam!’s work in basic science to that of “an alien piecing together the rules of basketball”: “There are rules, and instead of them being man-made rules . . . these are the rules of the motor system, these are the rules of the brain.”
Talking to Krakauer, I thought a lot about this everyday miracle—our capacity to consciously formulate goals, and to reach for them. Our goals can be as close at hand as a coffee cup or as coldly distant as the horizon beyond our deaths.
Candlelight worked itself up the wick, pasting twin red leaves onto John’s glasses. I blotted wax with a knuckle, newly grateful for my ability to take this tiny, voluntary action. Krakauer was explaining the pleasure he took in surprising the motor system into “giving up its secrets” with a well-designed experiment: “It’s amazing that there’s form and structure, truth, that it actually is discernible. Why did the motor system, the brain, turn out to have a logic? You know, it didn’t have to. You could imagine it being sort of chaotic, selfish, serendipitous, and perverse.”
His arms floated over their tapering shadows on the tablecloth. “Instead, it has structure, reproducible invariance. It’s amazing. I get tachycardic. I think, Oh, my God. That’s how the world devised this.”