Neuroimaging: Separating the Promise from the Pipe Dreams

On any given day you are likely to see a news report mentioning brain imaging. As I write this, a quick search of recent news stories yields the following headlines:

• “Justice May Be Hard-wired into the Human Brain” (New Scientist)
• “Brain Area Blamed for Stress Disorders” (RedOrbit.com)
• “School Bullies—Is the Amygdala to Blame?” (BrainBlogger.com)

Neuroimaging research clearly has captured the imagination of both the public and science writers. Given how far brain imaging has come in the last two decades, this is understandable. Functional magnetic resonance imaging (fMRI) has revolutionized our ability to safely image brain activity, and its broad accessibility has allowed researchers around the world to ask fascinating new questions about the mind and brain. At the same time, it is all too easy to leave the limitations and caveats of these methods out of the picture. This results in a common perception that overrates the power of brain imaging to explain everything from love and beauty to financial decision making.

One of the main limitations of neuroimaging is that conclusions based on studies of groups of people might not apply to individuals. This limitation becomes especially important when use moves beyond the realm of scientific generalities to the domain of individual health care decisions. The apparent power of neuroimaging can be overwhelming to the parent searching for an explanation for her child’s disruptive behavior, or to a child seeking answers about his parent’s memory loss. However, many proposed applications—particularly those relating to the diagnosis or treatment of psychiatric disorders—simply are not supported by evidence from research.

What Neuroimaging Can and Cannot Tell Us

Functional magnetic resonance imaging is a powerful tool. It measures blood flow in the brain, which increases when the neurons become active, thereby indirectly measuring their activity. For example, one might use fMRI to image brain activity while research participants are engaged in a mental task such as rehearsing a seven-digit number, and then compare the results with scans taken while the participants rest quietly. Using statistical methods to compare these images, we can determine where in the brain the difference is strong enough to be considered important. The colorful “activation maps” we see in news stories highlight these areas. We also can compare brain activity between different groups of people—for example, healthy individuals and individuals with schizophrenia—to determine which brain areas differ.

Neuroimaging research has provided fundamental insights into human brain function and mental disorders, and nearly every area of psychology and psychiatry has changed as a result. For example, drug-abuse researchers have used neuroimaging to isolate a set of brain systems where an abnormal response to rewards is associated with drug dependence. In principle, this research should lead to improved drug-dependence treatments that specifically target the isolated brain systems. It could also allow doctors to predict whether specific treatments will be effective, as researchers have created a biological index (or biomarker) for the function of these brain systems.

Neuroimaging research has also provided insights into brain development. Studies using fMRI have linked adolescence to increased activity in reward systems, while the development of prefrontal cortex functions, such as judgment and impulse control, lag behind. This difference helps explain the prevalence of impulsive and reward-driven behavior in adolescents. (See “The Teen Brain: Primed to Learn, Primed to Take Risks,” Cerebrum, February 2009.)

Despite this progress, neuroimaging leaves several questions unanswered. First, it is impossible to say whether increased or decreased activity in a particular brain region is “better” or “abnormal.” For example, some studies found that schizophrenia patients had increased activity in specific brain areas (such as the prefrontal cortex) compared with healthy individuals, while other studies found that patients had decreased activity in the same brain areas. These conflicting findings suggest that group differences may be specific to the tasks on which the groups were tested, and they cannot be broadly interpreted as reflecting “better” or “worse” function in the patients.

Second, we cannot assume that individual brain areas are uniquely responsible for specific mental functions, and thus that activation of those regions tells us what a person is thinking. A prime example of overreaching is the application of neuroimaging to marketing, an emerging field known as neuromarketing. On its Web site, the neuromarketing firm FKF Applied Research claims that its researchers have used fMRI to map a set of specific mental processes onto specific brain regions, and the firm can use those maps to determine how individuals respond to stimuli such as advertisements:

Over the past decade, fMRI has allowed us to “map” several key regions of the brain with a high degree of specificity. . . . A key part of that data is how the brain reacts in 9 well known and well mapped areas, such as the Ventral Striatum (reward), Orbitofrontal Prefrontal Cortex (wanting), Medial Prefrontal Cortex (feeling connected), Anterior Cingulate Cortex (conflict) and the Amygdala (threat/challenge). Using this mapped data, as well as data from other parts of the brain, we have developed a set of norms that help us understand what is happening inside a subject’s brain when they are exposed to a particular type of stimuli.¹

This claim relies on a simple logical error. The fact that the amygdala, for example, responds to threat does not mean that activity in this area signifies that a person is feeling threatened. That would be true only if threat were the only thing that activates the amygdala, and we know this is not the case.

Although this article focuses mainly on neuroimaging in the diagnostic context, other areas in which the implications of neuroimaging results are being cited range from love and happiness to creativity to decision making. Such applications often rely upon exactly the same kind of flawed logic. For example, an article in Esquire magazine (“Do I Love My Wife? An Investigative Report,” May 18) described the author’s participation in an fMRI study that examined how his brain responded to photos of his wife in order to “scientifically assess [his] love.” The researchers claimed to be able to determine the degree to which he feels lust, romance and attachment for his wife by looking at which regions of his brain were active while he viewed the photos. However, the regions the researchers identified are the same areas found in other studies to be associated with different functions; the claims about the writer’s love for his wife based on these scans are almost certainly overblown.

Diagnosis and Prediction Using Neuroimaging

Brain imaging has revolutionized psychiatry by providing compelling evidence for the biological basis of psychiatric disorders. A large and increasing body of research has shown that individuals diagnosed with psychiatric disorders show specific differences in brain structure and function when compared with healthy individuals. These findings lead us to imagine that the diagnosis of psychiatric disorders using brain scans is just around the corner. A closer look, however, demonstrates the problematic nature of this logical leap.

As an example, imagine a study that compares brain activity in response to monetary rewards in healthy individuals and in drug abusers. Researchers analyze the data to determine whether there is a statistically significant difference in activity between the groups. In this context, statistical significance means that such a difference in observed activity would be very unlikely (less than a 5 percent chance) if there truly were no difference between the groups.

Let’s say that activity in a particular brain region was greater in the drug abusers than in the healthy individuals. We cannot accurately predict who is a drug abuser based on the fMRI data alone, because some healthy individuals’ levels could be above the average for drug abusers, and some drug abusers’ levels could be below the average for healthy individuals. Without doing more specialized statistical analyses, we can’t reliably predict drug abuse from observations of brain activation.

A new set of techniques, however, offers the potential to classify individuals more reliably as healthy or not based on their brain activation. These come from a new field of statistics called machine learning. In the past few years, machine-learning analyses have been applied to fMRI data, and they have shown some ability to accurately diagnose individuals with psychiatric disorders. However, none of these analyses has been subjected to the rigorous tests that would be necessary to supplant standard diagnostic approaches.

Neuroimaging research also could completely change how we think about psychiatric disorders by rendering obsolete the idea that using discrete diagnostic categories such as schizophrenia or attention-deficit/hyperactivity disorder (ADHD) provides the best way to understand the underlying disorders. Today, these diagnoses are based on formal criteria, outlined in the American Psychiatric Association’s Diagnostic and Statistical Manual, that specify symptoms for each disorder. But these criteria have no basis in neuroscience. In fact, the psychiatric community has become increasingly concerned that traditional diagnostic categories actually obscure the underlying brain systems and genes that lead to mental health problems. In addition, a growing body of evidence indicates that many psychiatric problems lie on a continuum rather than being discrete disorders, in the same way that hypertension reflects the extreme end of a continuum of blood pressure measurements. Neuroimaging provides us the means to go beyond diagnostic categories to better understand how brain activity relates to psychological dysfunction, whereas using it to “diagnose” classical psychiatric disorders could obscure, rather than illuminate, the true problems.

Another potentially important application for neuroimaging is to determine the best treatment for individuals with mental health disorders. For example, some children with ADHD do not respond to the stimulant medications that doctors usually prescribe for this disorder. It would be useful to be able to predict which children will benefit from which kinds of medication. Unfortunately, evidence is not yet sufficient to support this kind of prediction.

Nevertheless, the use of brain imaging has been championed to optimize treatment of psychiatric disorders. For example, Daniel Amen, M.D., has clinics that charge several thousand dollars to perform single photon emission computed tomography (SPECT) scans—which involve the injection of a small amount of a radioactive tracer—on individuals with one or more of a wide range of disorders, with the promise of “tailor[ing] treatment to your brain.” With respect to one of these disorders, Amen says on his Web site that children with ADHD who have a “ring of fire” pattern of brain activity respond better to anticonvulsant or antipsychotic medications than to stimulant medications. Amen’s own research, however, reports accuracy of less than 80 percent in predicting treatment outcomes. And even this number is likely to be an overestimate, as the study in question—which involved 157 children overall—excluded 120 patients who did not show activation in a particular brain region.²

Given the scant evidence, it seems unlikely that the potential benefit on treatment outcomes justifies exposing children to even the small amounts of radiation from SPECT scans. The American Psychiatric Association affirms this opinion in a 2005 report: “[T]he available evidence does not support the use of brain imaging for clinical diagnosis or treatment of psychiatric disorders in children and adolescents.”³

Someday neuroimaging will provide a better means to tailor treatment for psychiatric disorders. Until proper clinical trials show its effectiveness, however, we must regard this powerful technology with healthy skepticism.

Evaluating Neuroimaging Research

Brain imaging will only continue to penetrate our 21st-century lives. Because it reminds us that mental disorders are biologically based, this development is positive. However, it is important for readers and consumers to understand how scientists study and apply it.

First, publication in a peer-reviewed journal is necessary for any research finding to be taken seriously. Peer review is a cornerstone of modern science, the gatekeeper that helps ensure that any published research finding meets accepted standards for scientific methodology and reasoning. The peer review process is not perfect, however, and inappropriate claims and inaccurate research sometimes fall through the cracks.

Papers in the most prominent journals are sometimes the biggest offenders when it comes to over-interpretation. Because these journals tend to publish findings of substantial general interest, authors often present strong—and potentially inaccurate—interpretations of their results. “Special issues” of peer-reviewed journals also are prone to publishing flawed papers. In such issues the usual peer-review process is often weaker, as the guest editor may be reluctant to reject solicited submissions.

Second, it is important that any research finding be replicated by multiple independent groups of researchers before it is used as the basis for decisions about medical treatment. This is especially true for studies using small sample sizes, which is often the case in neuroimaging research due to its substantial expense. Defining a sample size as “small” is difficult because it depends on the scale of the effect that is being measured, but for standard neuroimaging studies, any finding based on fewer than about 20 participants should be treated with skepticism.

Third, researchers must show how differences in brain activation—either differences between participant groups or differences resulting from some intervention—relate to behavior or cognitive function. Such brain activation differences are not, on their own, compelling. Meaningful examination of group differences requires that groups be tightly controlled so that the only difference is the one in question. This type of control can be very difficult in the context of psychiatric disorders. For example, if two groups are tested and one of the groups underperforms, this difference in performance can result in differences in brain activity that do not relate directly to a disorder. Likewise, when individuals are trained on a task and they become better at it, changes in brain activity could simply reflect the fact that they are performing better, rather than reflecting a fundamental change in how their brains are processing information.

Fourth, brain areas do not correspond uniquely with mental functions, as news reports about neuroimaging research often imply. While the activity of specific areas certainly is crucial to such functions as memory or fear, it does not follow that these regions are “memory areas” or “fear areas.” These areas are probably involved in other functions as well; conversely, other brain areas are probably involved in memory and fear. Individual brain regions never work independently. The brain is a complex, dynamic system, and the activity of any area can only be interpreted in the context of other active areas.

Finally, we must be aware—and wary—of the commercial interests of the scientists publishing neuroimaging research. For example, many studies have examined the potential role of fMRI in lie detection. Two of the groups involved in this research have licensed their technology to companies that are developing a commercial product. Before any such results are taken seriously, they should be replicated by independent groups that are free of any conflict of interest.

Neuroimaging has the potential to revolutionize how we view our own minds and brains, and how we understand the brain dysfunction that results in psychiatric disorders. However, we should keep in mind that these amazing pictures of brain activity are much more complicated than they initially appear, and their interpretation is never simple.
These methods hold the promise of providing guidance for individual medical treatment, but that promise has yet to become a reality.