Pavlov’s dogs, hearing a bell just before mealtime, learned to associate it with food so strongly that the mere sound of it would make them salivate. But what if the food had followed the bell only half of the time? And what if the probability of a meal after the bell had been utterly unknowable?
Outcomes in the real world often involve such risks and ambiguities, and in the past few years neuroscientists have begun to find out how the brain tracks them.
Uncertainty and risk
Given the seeming complexity of human versus animal decision making, “It’s amazing that there’s as much agreement as there is between animal studies and human studies,” says Duke neurobiologist Michael Platt, author of a recent review paper on uncertainty in Nature Neuroscience.
One issue of agreement concerns the ventral tegmental area (VTA) in what is called the midbrain. When a suitably conditioned human or other animal is presented with a cue that predicts a reward, dopamine-producing neurons in the VTA become active, their long fibers exciting neurons in a limbic region called the ventral striatum. This activity appears to encode the anticipated value of the reward.
Experiments in both animals and humans have shown that when the probability of the outcome is systematically varied, and the outcomes are not too far apart, the VTA and striatal responses tend to track the moving average or “expected value” of the outcome (its subjective value multiplied by its probability of occurring).
If the brain could handle uncertainty only to the extent of bundling it with value into an “expected value” quantity, we and other animals would be relatively indifferent toward uncertain, “risky” outcomes. Yet there is ample behavioral evidence that we are not.
“If offered a sure bet of $100 or a risky bet with the same expected value, say a 50-50 gamble on $200, most people will take the sure bet,” says Platt. “And in analogous situations [involving food instead of money] most animals will take the sure bet as well.”
How does the brain know the difference? Several recent studies have found that separate groups of neurons in the prefrontal cortex become significantly more activated as the uncertainty associated with two possible outcomes increases.
The insular cortex, often called the insula, is thought to encode another key uncertainty-related signal, helping to convey “how you might feel if you were to engage in one option but you don’t actually get what you expected,” says Martin Paulus, a psychiatrist and neuroscientist at the University of California, San Diego.
As Paulus notes, much remains unknown about the insula, including whether it represents only the “bad” outcome of a given set of choices, as has often been assumed, or whether it somehow also represents the “good” outcome. But in the broad activity mapping of brain-imaging studies, it appears on the whole to react more strongly to bad outcomes. “This system is primarily devised to maintain body integrity,” says Paulus. “If you feel good, you’re just maintaining your integrity, but if you are challenged, then that has to be acted on almost immediately. And that’s probably why it’s much easier to find a representation of the negative outcome than of the positive.”
The VTA and its downstream target, the ventral striatum, also appear to encode an uncertainty-related signal. Several animal and human studies in the past few years have suggested that some neurons in this network track a stimulus’s expected value with a short-term, “phasic” burst of activity, while others appear to track uncertainty with a more gradual, “tonic” rise in their baseline firing rate, reaching a maximum when uncertainty peaks—that is, when positive and negative outcomes become equally probable.
As Platt notes, some researchers argue that this tonic increase in VTA-striatal activity “kind of makes the risky alternative seem more rewarding.” A study in 2005 by Camelia Kuhnen and Brian Knutson at Stanford also found that the ventral striata of human volunteers showed more activity on functional magnetic resonance imaging (fMRI) when they chose riskier investments over safe ones. A separate study found that lesions to a part of the ventral striatum, the nucleus accumbens core, caused rats to become very risk-averse, suggesting again that the striatum encodes a key signal needed to undertake risks. “So I think there is some converging evidence,” says Platt, “although the skeptic in me wonders to what extent we can really pin down exactly what the ventral striatum is doing at this point, with the tools that we have.”
The major tool for research on live human brains is fMRI, but each “voxel” it resolves includes, at a minimum, hundreds of thousands of neurons. In animals, far more precise readings can be done, even of single neurons, using implanted electrodes, and such studies often reveal a finer structure than fMRI can resolve.
For example, there is evidence now, largely from electrode-based animal studies, that the striatum doesn’t respond uniformly to losses or gains; it responds with distinct loss-related and gain-related neurons, which may be mutually inhibitory. “It looks like there is a gradient, with more anterior and ventral regions [of the striatum] being reward dominant, and then more posterior and caudal regions being slightly more punishment-sensitive,” says Ben Seymour, a neuroscientist at the Wellcome Center for Neuroimaging at University College London. A study by Seymour and his colleagues in the Journal of Neuroscience last year was the first to find specific loss-sensitive neurons in mice.
Most of the studies in which animals or human subjects have been presented with probabilistic outcomes have cued their subjects to those probabilities in one way or another. In the real world, of course, the probabilities of different outcomes may not be known with such precision—or at all.
There is evidence that the brain tracks this “ambiguity” of outcomes too. A study by Caltech’s Ming Hsu and colleagues in 2005 found that the perception of ambiguity, as opposed to mere risk, correlated with activity in the amygdala, an emotion-processing center in the limbic system, and also in the orbitofrontal cortex. In the same study, individuals with damage to their orbitofrontal cortices were found to be relatively insensitive to ambiguity in their decisions.
A study in 2006, led by Scott Huettel in Platt’s laboratory at Duke, also found evidence that ambiguity-related correlations could be found in the prefrontal cortex. “Both studies did find that you could assign the representation of ambiguous and risky options to different neuronal populations,” says Platt, although the two findings disagreed somewhat as to which populations those were. (“It has a lot to do with how you analyze the data and maybe the precise way in which the task is structured,” he says.)
Researchers are working now to locate precisely the limbic and cortical centers where ambiguity and risk are separately tracked. Platt’s lab recently did a study in monkeys, the results of which have so far been presented at meetings but not yet published. “The upshot is the monkeys are ambiguity-averse just like people, and [again] we found very separate populations of neurons that signal either ambiguity or risk and not the other,” he says.