Scientists Measure ‘Unexpected Reward’ Response in Humans

by Jim Schnabel

June 4, 2009

Neuroscientists have known for years that animal brains respond more strongly to unexpected rewards or punishments than to those they expect. In theories of basic learning, this degree of unexpectedness or surprise is important because it represents the new, unforeseen information that the brain must somehow incorporate into its model of “what happens next.” Researchers have now made direct electrode recordings of dopamine-producing neurons that encode this “surprise” element in human brains.

“It’s great work,” says Kim D’Ardenne, a researcher at the Baylor College of Medicine who led a 2008 study in humans of a closely related group of neurons using functional magnetic resonance imaging. “To be able to record that kind of data from humans is priceless.”

Scientists normally cannot implant electrodes into people’s brains just to do research because the procedure is invasive and surgery is risky. In this case, the growing use of deep brain stimulation (DBS), a therapy for Parkinson’s and other diseases, opened an ethical window of opportunity.

When a person with Parkinson’s undergoes DBS surgery, the electrically stimulating wire is threaded into a region of the midbrain known as the subthalamic nucleus. To place it precisely, neurosurgeons triangulate the proper position by monitoring its signals using microelectrodes implanted in neighboring brain tissues. Since these electrodes already have to be in place—their risk is offset by a clear medical benefit—it is generally considered ethical to make use of them, briefly, for neuroscience experiments, given the informed consent of the patient. An experiment based on microelectrode recordings in epilepsy patients yielded a landmark paper last year on how memory works [see the news story “Neurons caught in the act of remembering”].

In the present study, reported in the March 13 issue of Science, a team led by Kareem Zaghloul, a neurosurgeon at the University of Pennsylvania Medical School, used DBS microelectrodes to study some of the neurons that participate in the brain’s “reward and motivation” system.

Some of the more important neurons that respond to unexpected rewards in animals are found in the dopamine-producing, midbrain structures called the ventral tegmental area and the substantia nigra. The latter lies just below the subthalamic nucleus, so Zaghloul and his team placed electrodes at this boundary to eavesdrop on substantia nigra neurons in ten people with Parkinson’s who were about to undergo DBS surgery.

While their neurons’ firing patterns were being recorded, the patients played a standard reward-learning game, featuring two decks of cards—one blue, one red—on a computer screen. They were asked to choose cards from either deck to determine which deck had the higher proportion of “reward” cards. When a reward card was chosen, the screen displayed an image of gold coins with a counter showing accumulated gains, and a speaker played the ringing sound of a cash register. Other cards brought losses. Using this feedback, they learned quickly to choose the deck with the higher reward probability.

The researchers took the choices each patient had made over time and fitted these to a standard model of reward expectation for a sequence of events. They used this model to classify each gain or loss as either expected or unexpected.

Zaghloul and his colleagues found that for unexpected gains, as compared with unexpected losses, one or more clusters of dopamine neurons near the implanted electrode increased its firing rate significantly during a crucial response interval after the gain or loss feedback was presented. For expected gains and losses, there was no significant difference in firing rates.

The study represents a milestone in the investigation of this basic learning system in the brain because it finally translates results from electrode recordings in rodents and primates to humans, Zaghloul says. As DBS is starting to be investigated to treat other conditions, researchers may be able to study neurons directly in other brain regions as well.

The results reported by Zaghloul and his team incidentally highlight some of the less visible effects of Parkinson’s disease, which include disorders of mood and executive functions such as attention. These functions all rely to some extent on reward-related dopamine signaling. “As more and more dopamine neurons die in patients with Parkinson’s, the reward-related signals that the dopamine system broadcasts to other parts of the brain are diminished,” says D’Ardenne.