“Out with the couch, on with the headset.”
As the meteors came down from the sky, my heart thudded in my chest. There was only one way I could save the town below: Reach out into the air, make a fist, and in doing so, set off an explosion. And then another. And another. How else can one be expected to defend a village?
This was the task given to me by Alex Miller, a computer scientist creating virtual realities for the neurology department at the University of Pennsylvania. Under the guidance of Dr. Branch Coslett, Miller’s lab is making programs for stroke victims trying to regain the use of a limb, amputees trying to lose phantom limb pain, and other people with mysterious, hard-to-heal conditions of the body and brain.
But in the meantime, I’m strapped into an Oculus Rift VR headset, with a Leap Motion tracking system attached to the front of it. The Leap Motion is, well, magical: it scans the area in front of it, registers where my hands are, and then projects those hands into the game. The experience is profoundly immersive: when I move my head left or right, the view in the game moves accordingly, and if I open my hand or close my wrist, the same happens in-game, in real-time. There are many possible medical applications: the game is recording all of my movements, creating what would be a hyper-detailed tracking of rehab progress over time, and if Miller so chose, my in-game left hand could be a representation of my real-life right hand—a fun trick for an able-bodied person, but if I had lost my left hand, my brain seeing an intact left hand in game could actually ease phantom limb pain.
Just a few minutes earlier, Miller was strapping sensors to my thighs and knees. I’m in yet another world, and with my virtual feet underneath me. I’m on a platform in a desert, it’s twilight, and I have a puzzle to solve: make it from my spawning point to a glowing goal across the way. There are any number of pits I need to avoid falling into, and in classic game fashion, the solution is to push crates into them. Seated in my chair, I make a slow, dragging step to move forward, and sweep my foot left or right to turn. The sensors strapped to me are set up so that if you had lost your leg below—or above—the knee, the electronic signals sent by the muscles would be detected, and you’d make those same movements in the game. So even for a body that is injured in real life, it can be intact in the game, and neurologically speaking, it doesn’t make that much of a difference.
“That’s the killer use case of virtual reality,” Miller explains. The brain is surprisingly easy to fool, and it will believe that the hands and feet in a game are your own, with potentially huge medical consequences. “It’s really about illusion,” he says: you manipulate what a patient sees in their virtual self and their virtual world, and their brains will literally incorporate these things into the body image. While still early, results indicate that using the Penn neurology games does indeed reduce the intensity of phantom limb pain.
The idea of virtual reality has been around for decades. The French dramatist Antonin Artaud coined “la realite virtuelle” in 1938 to describe the temporary world created by theatre. The 1960s saw the Sensorama, an arcade cabinet that played 3D movies along with stereo sound, wind and smells on head-mounted displays. Movies like Tron, The Lawnmower Man and The Matrix all help make VR a household term. And the 1990s saw a boom in VR arcades, with game console manufacturers making early bids, too—shoutout to Nintendo’s Virtual Boy. The first examples of medical VR started showing up then, too—like a demo of gastrointestinal surgery. But, according to many researchers Thrive Global spoke to, we’ve entered a new era of VR in just the past few years.
With the Oculus, the HTC Vive, and other VR setups becoming available, the price of setting up a VR lab has cratered: Betty Mohler, a researcher at the Max Planck Institute for Biological Cybernetics, tells Thrive Global the cost has fallen a hundredfold. And it’s getting even cheaper, as low-cost options like the Google Daydream start rolling out. “With affordable, high-quality virtual reality devices hitting the market for the first time, the future seems suddenly imminent,” Oxford psychiatrist and VR specialist Daniel Freeman tells Thrive Global. “VR could become the method of choice for psychological treatment — out with the couch, on with the headset.”
It’s a potential that the investors have seen, most famously with Facebook’s $3 billion purchase of Oculus in 2014. More recently, with the growing hype surrounding MindMaze, a VR startup out of Switzerland with a valuation already north of $1 billion. The company’s stroke rehab treatments were introduced into European hospitals in 2013, and the company announced entry into the US market this year. Since VR is so stimulating, patients are more likely to do their rehab, and according to one company report, a full 100 percent of patients forgot they were in the hospital while doing their VR rehab.
VR is simultaneously neurological and psychological: it has applications with disorders of the body, like phantom limb pain, and conditions of the mind, like PTSD, anxiety, and paranoia. Unlike any other technology before it, VR gives the user a direct sense of embodiment, what University of Barcelona pioneer Mel Slater refers to as “presence.” It’s not just another medium in a long line of media: virtual reality directly accesses people’s sense of self—these hands are mine, my own brain thought, as I pawed at the meteors. Rather than flatly watching, you’re immersed in the virtual world.
The thing about conventional “talk” therapy is that all the therapist and the client can really do is remember and imagine: you might get tips about how to keep your family from driving you crazy the next time Thanksgiving rolls around, but your shrink can’t place you at the dinner table. That all changes with virtual reality: with the right software, a therapist can put you in the places you have come to fear the most. People with arachnophobia get anxious around spiders, those with paranoia are afraid of social situations, and people with PTSD get triggered by cues linked to their trauma.
“Difficulties interacting in the world are at the heart of mental health issues,” Freeman, the Oxford psychiatrist, explains. With VR, you can repeatedly experience those feared situations—at a just-tolerable dose—and learn to overcome them. “The beauty of VR is that individuals know that a computer environment is not real, but their minds and bodies behave as if it is real,” he adds, which allows people to more easily face fearful situations, and then experiment with how to approach them. “This learning then transfers to the real world,” he says.
His lab has done lots of research around paranoia, a condition that affects around 1 or 2 percent of the population. In a 2016 paper in the British Journal of Psychiatry, Freeman and his team dropped people with paranoia into public situations, like standing in an elevator or commuting on a subway car. The subjects wore headsets and could walk around a room, rather than using a controller. Every time the patient entered one of these levels, they’d encounter more people in that space. After doing just 30 minutes of VR, about half of the patients no longer felt severe paranoia at the end of their testing day, and when they went into real social situations, they felt less distressed.
This is an early, promising, and high-tech version of an old psychological technique—exposure therapy, where you give people incrementally larger hits of the situation they fear, whether it’s flying or public speaking or whatever, and they gradually become less fearful as they grow more accustomed to it. While the studies on VR therapy haven’t been uniformly consistent, Freeman’s new review of the literature finds that VR versions of exposure therapy help people with phobias of heights and animals, those with social anxiety, and people who fear public speaking, among other conditions. “As patients are helped to face the situations they fear, they learn that they are safe,” he explains. If someone spends more time in front of higher and higher virtual cliffs, they discover nothing awful happens to them, and their fear of heights starts to subside. “The fact that the person knows the scenario is artificial makes no difference,” he says. “The mind and body respond as if it were real.”
Like if, for instance, you were in a combat zone.
Between eight and 18 percent of American veterans returning from wars in Iraq or Afghanistan have some form of PTSD. Stigma against seeking care remains, and among those who do, the results aren’t great: a recent editorial in the Journal of the American Medical Association—Psychiatry called out an “ongoing crisis” in PTSD care. An average of 28 percent, and as many as 40 percent of patients drop out of treatment programs, and those who do finish have no guarantee the symptoms will decrease.
Into this comes Bravemind, a virtual reality therapy program lead by Albert “Skip” Rizzo at the University of Southern California’s Institute for Creative Technologies, which has helped thousands of veterans.
The goal is to bring the patient back into the moment they were traumatized: the clinician will place the patient in the driver’s seat or the back of the humvee or the front of a foot patrol, wherever they were, set the time of day or night, and dial up or down the number of people, explosions, and the like. Bravemind’s fourteen environments were designed to bring patients back to the places they were traumatized: an Iraqi suburb, an Afghan village, a remote mountain base, a highway checkpoint.
“The clinician is setting up the stage and pushing them to go deeper and talk about their emotions,” Rizzo says. Once they put the headset on, feel the platform they’re sitting on rumble, and feel the thump of the subwoofer mimicking an engine, they’re back there.
While the patient knows, intellectually, that they’re in a therapy office, and talking with a therapist they trust, the older, emotional, threat-sensitive part of the brain—which becomes hyper-vigilant in PTSD—thinks that it’s back in a warzone. “That’s where the magic comes in,” Rizzo says. “We can activate the areas of the brain we want to target to foster therapeutic effect.” While these brain areas do indeed get freaked out by the simulation, they also learn that they’re safe. Memories are pulled up, confronted, and processed. In clinical trials, this virtual reality exposure therapy has reduced symptoms for about three-quarters of patients. Early research also shows the promise of VR for military sexual trauma, and Rizzo has plans to build versions that would help civilians who experience violent trauma, like firefighters and police, who also have an epidemic of PTSD, as well as victims of terrorist attacks. Indeed, there have already been promising VR-based interventions with people who had PTSD following the World Trade Center attacks.
VR-based exposure therapy may be even more effective in outpatient settings: a new study of a three-week intensive program found that two-thirds of veterans no longer suffered from PTSD at the end of the intervention—a mark that held at six-month follow-up.
VR isn’t limited to the way people subjectively experience their bodies or process cues in the world, but also in how they view their own bodies. Betty Mohler, the Max Planck researcher, says that a lot of this has to do with avatars—that is, our virtual representations of ourselves in virtual reality.
Her team has done intensive avatar-making with clinical populations of people with anorexia: they stand in front of 22 medical grade cameras, snapping photos of their bodies. Contrary to popular belief, eating disorder patients usually don’t have some sort of visual malady where they can’t tell how thin they are; they see that they’re thin, and they want to be thinner.
At the center is the 3D body scan of an experiment participant, to the left and right are weight manipulated bodies based on a statistical model derived from 2100 scans of women. Image courtesy Ivelina Piryankova & Max Planck Institute for Biological Cybernetics & Intelligent Systems.
Mohler’s software can visualize what people would look like if they were thinner or heavier. So the idea is to do another kind of exposure therapy: render patients in a body at a healthier weight, and help them grow accustomed to that. It’s helping people become comfortable in a new skin, as it were, before they get there.
The future of VR is one of ever greater immersion. Humans, like every other ape, are visual creatures—that’s why your brain will so often take to be what it sees as what really is. But with advances in VR, more senses will be involved to make the experience even more immersive: to date, the best rigs allow the user to use their own movement to move in the virtual world. Coslett says that for years, science thought these different senses were distinct from each other, but complementary. And now it’s being realized that touch, taste, sight, and the rest all get bundled together into a single “percept.” The brain is readily convinced by synchronization: if things happen at the same time, then they must be related.
Imagine a VR vest that targeted your internal organs. Image courtesy of Frontiers in Human Neuroscience.
The more multisensory an experience is, the more immersive it will be. That’s what Bravemind is getting at with a rumble beneath his veteran subjects and a thumping subwoofer; Rizzo is even throwing smells at his users. Other researchers are proposing vests outfitted with vibrating transducers that will target your stomach and inner ear, so that your interception—or the awareness of interior organs—is targeted, too. Engineer Katherine Kuchenbecker, who started Penn VR lab with Coslett and Miller, has also developed a touch feedback system for robotic surgery, so doctors doing surgery from afar get the kinesthetic and tactile cues they get in conventional operations. This helps surgeons figure out much faster if they’re making a mistake or injuring the patient, and learn new surgeries faster. The same principles can be carried over to VR: if you get enough multisensory feedback in VR, then it will be more like the experiences you’ve already had in real life, like your intact body part before you lost a limb. The more lifelike it is, the more you’ll be able to help heal the psyches of soldiers or abate the pain of amputees. “You’re basically going through a loophole in the brain,” she says.