New Twist on ‘Reward Response’ Model in Brain

by Jim Schnabel

November 9, 2009

New research has unsettled old assumptions about a class of brain cells believed to be essential for the ability to learn from life’s rewards and punishments. Teams of neuroscientists in Britain and the United States have found evidence suggesting that these reward-sensitive midbrain dopamine neurons, contrary to widespread belief, do not all function in the same way.

“There’s been this ongoing debate about the role dopamine neurons are playing in reward and aversion, and one resolution is that there are just different dopamine neurons doing different things,” says Mark Ungless, a neuroscientist at Imperial College, London, who led one of the research teams.

For decades researchers have investigated the functions of midbrain dopamine neurons, whose outputs feed into brain structures involved in learning, action and emotion. Many studies have shown that these neurons are excited by unexpected rewards, such as food, and also by cues that predict those rewarding events. The greater the value of the reward, the more these neurons pump dopamine into other regions of the brain. In one of these regions, the striatum, this dopamine signal is thought to serve as a key learning signal, adjusting motivational priorities so that the animal is more likely to seek out that reward in the future.

When the stimulus is negative, such as an expected reward that fails to materialize, or an unexpected punishment, these same midbrain dopamine neurons briefly slow their firing below the baseline rate, apparently sending an unlearning signal into the network. The differences in how this circuitry functions in individuals may account for major aspects of personality, such as impulsivity, novelty-seeking and susceptibility to addiction.

But not every experiment shows midbrain dopamine neurons encoding the value of a stimulus. Some studies have suggested that the neurons instead are signaling the absolute importance, or saliency, of an event, regardless of whether it carries a positive or negative value.

Hoping to resolve the matter, Ungless’s team recorded signals in two closely situated areas of the midbrain while anesthetized rats were subjected to a mild foot shock. In one area, dopamine neurons were inhibited, as expected under the current model. But in the other area, dopamine neurons were excited by the shocks. In a report published in January in the Proceedings of the National Academy of Sciences, Ungless’s group suggested that the finding might reflect the presence of two functionally separate groups of dopamine neurons, one encoding value, the other encoding saliency.

In a study published in June in Nature, Okihide Hikosaka and his postdoctoral researcher Masayuki Matsumoto, at the National Eye Institute, part of the National Institutes of Health (NIH) in Bethesda, Md., appeared to confirm this finding with a study in macaque monkeys. The researchers recorded the electrical activity of midbrain dopamine neurons while the awake, alert monkeys received sips of apple juice or  unpleasant, abrupt puffs of air on the face.

Hikosaka and Matsumoto’s recordings were from different midbrain areas than those probed by Ungless’s team, but the NIH researchers also found evidence for both the classic “value type” neuron that is excited by rewards and inhibited by punishments, and a “saliency type” that is excited by both unexpected rewards and punishments but stays quiet when nothing interesting happens.

To Hikosaka, the evidence for a saliency circuit suggests that midbrain dopamine neurons don’t merely encode a learning signal about a stimulus, as had been thought, but might also direct immediate action to avoid or enjoy the stimulus.

“The saliency-related dopamine signal may either facilitate or suppress ongoing or upcoming behavior,” he says. “So if the salient event is expected, the animal needs to do something. How the animal differentiates between the two actions for good things or bad things we don’t know yet.”

Brown University neuroscientist Michael Frank, a dopamine-midbrain researcher not involved in either study, cautions that more research is needed to back up these results. “There might not be just one simple story for how dopamine neurons function,” he acknowledges, but he would like to see more evidence that the apparent saliency neurons are excited by truly negative stimuli as well as by positive stimuli. Whether these neurons and the classical “value” neurons send their conflicting signals to the same downstream brain areas also needs to be clarified: “It’s not clear how those downstream areas would be able to detect which kind of dopamine cell fired in which direction, so that’s a quandary,” Frank says.

In addition, there might even be more kinds of dopamine neurons in midbrain, reward-sensitive regions, Ungless notes. “There might be three or even more different groups of dopamine neurons that are encoding slightly different things,” he says. Acceptance of this multi-functionality would be “a major first step” in understanding the brain’s reward response networks, but he expects years to pass before neuroscientists unravel these networks’ full complexity.

“If you’d asked someone 20 years ago what dopamine neurons do, they would have told you that it’s pretty well established that they don’t do anything interesting, that their firing activity is really boring and that they show some responses to some stimuli but nothing special. In the last 20 or 30 years that view has been pretty dramatically changed,” Ungless says. “And I wouldn’t be surprised if in the next 10 to 20 years, our view changes very dramatically again.”