neuromorphogenesis:

“Pop, Pop, Pop.” She Heard Her Brain in Action

In November of last year, Jan Scheuermann did something she never thought she would do again: She fed herself a piece of chocolate. For the last decade Scheuermann, 54, has been a prisoner in her own body. She suffers from a mysterious degenerative disorder that attacks the nervous system, severing the connections between the brain and muscles. Now a quadriplegic, Scheuermann has no movement below her neck. She can’t move her limbs, let alone grasp, move, or hold anything. Until she’s hooked up to a brain-computer interface (BCI).

BCI is a technology that can plug in to neuronal activity in Scheuermann’s brain, interpret it, and translate the signals into action—specifically, moving a robotic arm that she calls “Hector” to grasp objects and bring them to her—as though it were a functional part of her body. Dr. Andrew Schwartz, one of the field’s pioneers, is testing the system in clinical trials run out of his lab at the University of Pittsburgh. 

BCIs have the potential to give people with limited mobility increasing independence. In the future they might restore the power of speech to victims of severe strokes, as well as offer radical remedies for mental illness and Alzheimer’s disease. Yet BCIs can also be used in invasive ways that compromise individual freedom, such as an infallible lie detector and robotic super soldiers. Both technologies are based on the same scientific breakthroughs, and both take scientists one stop closer to decoding the brain.  But scientists are finding that the brain does not give up its secrets easily.

Ever since the fourth century B.C., when the first physician, Hippocrates, identified the brain as the seat of intelligence, scientists have been trying to understand its inner workings. The first step toward applied neurology was made in the 1920s with the discovery of a tool to measure electrical activity, called an electroencephalograph machine. Discovering how to track electrical impulses allowed scientists to peer into the skull without drilling holes. In the 1980s, the advent of the magnetic resonance imaging (MRI) machine made it possible to measure and record brain activity in real time, effectively ushering in the golden age of neuroscience. Today, the goal of BCIs is to go one step further and create neural devices that are portable and user-friendly.

One of the fathers of BCIs, Director of the Institute for Brain Science at Brown University, John Donoghue says “the goal is to provide a route from the brain to the outside world for people who need it.” He developed an early BCI called BrainGate, on which Schwartz’s Hector is based. He also warns that we are still far from creating a direct interface with the human mind. Thought-reading robot servants will not be available at your local Best Buy anytime soon.

That would require technology that is sensitive enough to cut through the billions of electrical impulses which neurons (and groups of neurons) interchange per second and pick out the right ones. In other words, it would need to read the intention of the user, which is not something that a BCI can do just yet, as Jan Scheuermann has found out in her year and a half training with the prosthetic arm.

The brain does not give up its secrets easily.

In order to join the clinical study, Scheuermann first underwent a rigorous screening process and then a surgical procedure that implanted a pair of arrays, each with nearly 100 tiny electrodes, on the cortex (the surface of her brain). Penetrating 1/16 of an inch into her brain, each electrode can read the firing of a single neuron—an astonishing feat of miniaturized detection. The attachment point, where the cable connects to the array, is kept sterile and protected by a bandage when not in use. When Scheuermann is hooked up to the system, a thin wire connects to a thick cable that runs to a cart of electronics topped by a laptop, and thence to the large robotic arm.

Within days of recovering from the operation, Scheuermann, whose affliction has not dimmed her resolute spirit, was working with Schwartz’s machine, learning to control the system.  Four hours a day, three days a week, over months in the summer and fall, she practiced on the system, trying to control its actions. Clearing her mind of distractions, she concentrated on simple movements: “Move the arm to the left.” “Move the arm to the right.”  Progress was steady. After several weeks, she could move the arm in seven degrees of freedom, rotate the hand, and grasp a small item and bring it to her face. 

“Pop, pop, pop. It’s working,” she recalled in a phone conversation about her experience with the BCI trial. “It was like Rice Krispies.”

“Thanks to electrodes so finely tuned that they can pick up the signals from individual neurons,” says Donoghue, “we can see at a fine level how the brain is behaving in certain circumstances, how it’s thinking, and use that knowledge in many different applications including controlling a prosthetic.”

In effect, Donoghue explains that BCIs have to behave like a brain. They discern the signal for “move the arm to the left” from the noise of the billions of other neurons firing through an algorithmic framework of Bayesian probabilities that uses prior beliefs and tendencies to infer conclusions about the state of an uncertain system. The human mind seldom operates in absolutes; the signal for “Move to the left” may be a probability curve spread across thousands of neurons. BCI uses this data in order to calculate the desired action and act on it. 

Thought-reading robot servants will not be available at your local Best Buy anytime soon.

The new level of sophistication has come not just from machines that can more finely read the neurons’ noise; it also comes from the brain’s neural plasticity—its ability to change, to form new pathways and assemble new ensembles, in response to external experience and changing environmental conditions. As the machine responds to the brain and the brain controls the machine, BCIs become a co-adaptive hybrid. There is a feedback loop in which the machine learns to recognize the desired firing patterns even as the brain learns how to more effectively manipulate the device. The BCI system is self-optimizing even as Scheuermann learns to adjust her own thoughts to maximize her control of the system.

“The system is adapting to the user and vice versa,” explains Rajesh Rao a professor of computer science at the University of Washington and a BCI researcher. “It keeps changing the way it works with you so it can change its own mappings from the brain to the actions it executes.”

Ultimately, scientists are finding, the brain does not distinguish whether it’s moving a finger, angling a robotic prosthesis—or piloting a battleship.  The neural patterns are the same, and the brain—resourceful, pragmatic, adaptable to the tools at hand—learns quickly to assimilate the external machinery as if it were an extension of the body.

BCI’s capability of mentally powering another object also has potential for technology that intrudes on individual freedom. This was confirmed in late August, when Rao used a BCI to control one person’s hand with another’s mind. Wearing a cap that was studded with electrodes and an electroencephalography (EEG) reader, Rao imagined himself moving his right hand to fire a weapon in a video game on his screen. Across campus, the right hand of his colleague, wearing a similar array and connected via the Internet, moved. 

It was just a finger-twitch, but it portended much more.  The Rao experiment—which was publicized before being published in a peer-reviewed journal—gave rise to some dismissive, not to say derisive, reactions from other scientists. It also tapped into the deeper fears of BCI skeptics: that controlling external devices with the mind could lead to external control of the mind. After all, it is not a far leap from reading neural ensembles to manipulating them. 

At the University of Southern California, biomedical engineering professor Theodore Berger is using BCIs to actually improve cognition. Berger’s research centers on how the brain forms long-term memories—and how implantable chips can mimic that function. Last year, Berger and his team showed that rats with memory impairment could retrieve memories with the help of a BCI that “downloads” the previously recorded neural firing pattern associated with a specific memory—such as pulling the correct lever to obtain a piece of food. Essentially, Berger has given the animals an artificial hippocampus, laying the groundwork for creating technology that downloads false memories.

The human mind seldom operates in absolutes; the signal for “Move to the left” may be a probability curve spread across thousands of neurons.

A similarly invasive approach is being tested by the Defense Advanced Research Projects Agency (DARPA). A device known as the Cognitive Technology Threat Warning System places a soldier, equipped with the same skullcap that Rao used to get a finger to move across the University of Washington campus, before a large video display. As the soldier eyes the screen, her brain fires in a particular pattern when it recognizes an anomaly on the horizon that might point to a potential threat. The BCI recognizes it and can then magnify it for the human user—focusing her attention on it and asking if it warrants further analysis. 

This technology capitalizes on the brain’s unparalleled ability to notice patterns and a computer’s ability to develop them. It is not hard to envision the next logical step: a weapon that acts on the soldier’s threat-detection response and takes action against any perceived threat.

And what if, in addition to allowing the computer to act on the soldier’s responses, it could influence the responses as it magnifies and develops them? That’s the potential of the experiments in brain-to-brain communication being carried out by Rao and others. By reversing the flow—from interpreting signals to introducing them—scientists are starting to dive into murky waters.  Research has already demonstrated that the brain activity of sociopaths is distinctive at an early age; what is to prevent us, for example, from manipulating the neurons of a 4-year-old to prevent the murders she might commit at age 25?  If we can use neurology to control psychology, where are the limits?

These experiments might lead some to believe that BCIs will lead to a complete model of the mind, which can be used for medical breakthroughs or dystopian mind control. Yet what holds scientists back is that what counts as a “thought” is much different from Jan Scheuermann’s instruction to Hector to “move 5 inches forward” or DARPA’s Threat Warning System that looks for a specific group of neurons firing.  

Scientists say that the real brain decoding—the ability to read actual thoughts—would entail a computer that can recreate human feelings, memories, and desires. How can one define the myriad fleeting thoughts that an individual experiences and transcribe them? Discerning that level of intention will require new ways of decoding brain signals that are collected across larger swathes of cerebral geography. 

“Things start getting fuzzy: cognition, learning, memory, you can’t define those very clearly,” Schwartz remarks. “If someone told me, put electrodes in the brain and find a thought, well, you’ll have to tell me what that looks like before I know if I found it.”

“BCIs require a more detailed understanding of these network interactions, but there’s this big hole in the middle, at the mesoscale: For brain science to advance, we need to attack the problem at that level, and form a bridge, and we need the tools to do it,” says Donoghue. 

What is to prevent us, for example, from manipulating the neurons of a 4-year-old to prevent the murders she might commit at age 25?

Researchers at the Behavioral and Clinical Neurosciences Institute, at Cambridge, published a paper in 2009 that argued for a new way of looking at brain function and cognition. They claimed that the brain’s structure is fractal, in the sense of exhibiting near-infinite levels of complexity, each replicating the levels above and below. This means that it would be impossible to create a model of how it functions.  

To truly parse the neural patterns that lead to the composition of the “Ode to Joy” involves many, many neural circuits acting in self-organized systems that might not happen in any one location in the brain. They might be distributed across many areas. Recovering, or reproducing that dense interplay of electrical impulses remains beyond the reach of our most finely tuned neuro-imaging systems. What’s needed is an entirely new way of conceiving the brain at work.

Those challenges were underlined in April, when President Obama announced the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) initiative, a $100 million program to more completely map the brain. Announcing the new funding, Obama compared the BRAIN initiative to the Human Genome project: a long-term program to open up new vistas in medicine and human biology. 

Some in the scientific community believe that BRAIN and its correlated mapping efforts are not only futile, but also misguided. No matter how closely you track an electrical impulse through the mind’s critical network, you are no closer to “seeing” how the face of a child affects her father, or unraveling the tangle of emotion and intellect and memory behind Beethoven’s later works, or understanding the motivations of a psychopath. 

No one has yet looked at a living human brain and verbalized its ideas without the decoder of the human mind. BCIs will bring inestimable benefits to people trapped in immobile bodies, giving speech and action back to people with severe spinal injuries or disorders. Those accomplishments more than justify whatever risk of misuse the technology entails. But in probing the brain’s complex mysteries we are not only playing God; we may well be on a pointless search for the ineffable, a high-tech hunt for a chimera.