Summary: While neuroimaging may be a standard in neuroscience and psychology research, a new study says researchers are massively underestimating how large the study sample must be for a neuroimaging study to produce reliable findings.

Source: University of Pittsburgh

What does it take to know a person?

If you’ve seen how a friend acts across different domains of their life, you might reasonably say you know who they are. Compare that to watching an interview with a celebrity — maybe you can claim some knowledge about them, but a single observation of a stranger can only tell you so much.

Yet a similar idea — that a lone snapshot of a brain can tell you about an individual’s personality or mental health — has been the basis of decades of neuroscience studies.

That approach was punctured by a paper in Nature earlier this year showing that scientists have massively underestimated how large such studies must be to produce reliable findings.

“The more we learn about who we are as people, the more we learn that, on average, we’re much more similar than we are different — and so understanding those differences is really challenging,” said Brenden Tervo-Clemmens (A&S ’21G), now a postdoctoral fellow at Massachusetts General Hospital and Harvard Medical School who co-led the multi-institutional research as a clinical psychology PhD student at Pitt.

At the center of the research is MRI (magnetic resonance imaging) brain scans. While invaluable for diagnosing brain conditions, they’ve also been used by researchers to draw links between a person’s brain structure and aspects of their personality and mental health.

Tervo-Clemmens and his colleagues call this technique brain-wide association scans, or BWAS, in a nod to “GWAS” studies that attempt to decipher the often-tiny effects of genes from massive datasets (as seen in dubious science headlines announcing “a gene for depression” or “a gene for intelligence”).

“The approach is similar: Here’s one profile of you biologically, how well can we determine the complexity of your human experience?” Tervo-Clemmens said. “And the answer is, usually not very well.”

A typical study of the kind would include somewhere around 25 participants, due in part to the high cost of running scans. But, Tervo-Clemmens and his colleagues showed, scientists would need to scan the brains of more than 1,000 to be confident that the connections they find aren’t just a statistical mirage.

Reaching that conclusion required getting a far broader view of the field than was possible until recently. Along with colleagues at a number of institutions as well as his advisor, Pitt Professor of Psychiatry Beatriz Luna, Tervo-Clemmens combined three recent publicly available studies that together included MRI data from around 50,000 participants.

Using this massive body of information, the team simulated the process of science, selecting groups of the scans at random as if they were patients recruited to a study. By repeating that process over and over, the researchers could figure out how likely it is that any given number of scans would produce a misleading result simply due to chance — and how many participants it takes for a study to be reliable.

Not every investigation requires 1,000 brain scans, they showed. “If the goal is just to understand something like the general organization of the brain, we sometimes only need 10 to 20 participants to do that,” Tervo-Clemmens said. It’s only because a single brain scan reveals so little about a person’s personality and mental health that researchers need a massive amount of data before these complex traits begin to reliably stand out from the statistical noise.

Amplifying that problem is a well-known bug in 21st century science: Researchers are often rewarded for publishing results that show exciting new connections, rather than less glamorous findings suggesting the absence of a connection.

The latter results are less likely to be published and more likely to languish on a hard drive. So not only are small imaging studies more likely to “discover” a link that isn’t actually there, but those same misleading studies also receive a disproportionate amount of attention.

Tervo-Clemmens is quick to note that the Nature paper wasn’t intended to call out the whole field. Neuroscientists and psychologists have successfully tackled questions about personality and mental health using a variety of other techniques. And brain scans on their own are very effective for diagnosing conditions and mapping out the broader picture of how brains work. It’s when scientists combine the two, reducing the complexities of a person into a single image, that they fall short.

“We can count on less than a hand the number of these studies that have held up under scrutiny and are really driving treatment,” he said. “In my own area, one study might show that increased function of a particular brain region is related to more symptoms, but you can find, almost without question, another study showing the opposite effect.”

How Not to Use Brain Scans in Neuroscience

A typical study of the kind would include somewhere around 25 participants, due in part to the high cost of running scans. Image is in the public domain

Although he now focuses mostly on psychiatric and substance-use disorders in adolescents, Tervo-Clemmens doesn’t quite fit into any one box as a researcher. “I’m kind of a psychologist, and I’m kind of a statistician, and I’m kind of a neuroscientist,” he said. It’s that perspective, he explains, that helps him do the kind of broad critical research like this current study, along with his boundary-crossing education at Pitt.

He saw patients as a PhD student in clinical psychology while also training in cross-disciplinary programs like Center for the Neural Basis of Cognition, experiences he credits as encouraging breadth in research. “I think that level of integration is what makes Pitt so awesome as a graduate student,” he said.

The result was a study that’s already produced a stir among other scientists. An instant classic, the paper and its pre-publication version have already been cited by more than 250 other scholarly works.

So where does that leave the field?

First, Tervo-Clemmens said, it’s necessary to re-examine the smaller studies of the past to see if their results hold up to further examination. As for future research, one solution would be to simply supersize brain-scan studies of complex behavior so they stand up to statistical scrutiny. But there’s another possible way forward, where researchers find ways to study patients over time and across contexts to get a fuller sense of their identities.

“We need to be aligning our research to how we generally think and understand human beings,” said Tervo-Clemmens. “That’s a challenge of cost and economy. But I also think it’s one that will ultimately be worth it.”

It’s like growing pains for a line of research that’s only a few decades old: Stressful and full of uncertainty, but also a sign that the field is heading in new and exciting directions.

About this neuroimaging and neuroscience research news

Author: Nicholas FranceSource: University of PittsburghContact: Nicholas France – University of Pittsburgh
Image: The image is in the public domain

Original Research: Open access.
“Reproducible brain-wide association studies require thousands of individuals” by Brenden Tervo-Clemmens et al. Nature


Abstract

Reproducible brain-wide association studies require thousands of individuals

Magnetic resonance imaging (MRI) has transformed our understanding of the human brain through well-replicated mapping of abilities to specific structures (for example, lesion studies) and functions (for example, task functional MRI (fMRI)). Mental health research and care have yet to realize similar advances from MRI.

A primary challenge has been replicating associations between inter-individual differences in brain structure or function and complex cognitive or mental health phenotypes (brain-wide association studies (BWAS)). Such BWAS have typically relied on sample sizes appropriate for classical brain mapping (the median neuroimaging study sample size is about 25), but potentially too small for capturing reproducible brain–behavioural phenotype associations.

Here we used three of the largest neuroimaging datasets currently available—with a total sample size of around 50,000 individuals—to quantify BWAS effect sizes and reproducibility as a function of sample size. BWAS associations were smaller than previously thought, resulting in statistically underpowered studies, inflated effect sizes and replication failures at typical sample sizes.

As sample sizes grew into the thousands, replication rates began to improve and effect size inflation decreased. More robust BWAS effects were detected for functional MRI (versus structural), cognitive tests (versus mental health questionnaires) and multivariate methods (versus univariate). Smaller than expected brain–phenotype associations and variability across population subsamples can explain widespread BWAS replication failures.

In contrast to non-BWAS approaches with larger effects (for example, lesions, interventions and within-person), BWAS reproducibility requires samples with thousands of individuals.

TECH NEWS RELATED

SpaceX capsule heads to space station ferrying NASA crew and Russian

CAPE CANAVERAL, Fla. — A SpaceX rocket soared into orbit from Florida on Wednesday carrying the next long-term International Space Station crew, with a Russian cosmonaut, two Americans and a Japanese astronaut flying together in a demonstration of U.S.-Russian teamwork in space despite Ukraine war tensions. A high-ranking official ...

View more: SpaceX capsule heads to space station ferrying NASA crew and Russian

Documentary featuring Professor Sara Seager wins Emmy Award

“The Hunt for Planet B” follows Seager and others on their search for extraterrestrial life; three other nominated films feature MIT affiliates.

View more: Documentary featuring Professor Sara Seager wins Emmy Award

Petting Dogs Engages the Social Brain, According to Neuroimaging

Summary: Interacting with dogs leads to greater activation in the prefrontal cortex, an area of the brain associated with emotional and social processing. Source: PLOS Researchers led by Rahel Marti at the University of Basel in Switzerland report that viewing, feeling, and touching real dogs leads to increasingly higher levels ...

View more: Petting Dogs Engages the Social Brain, According to Neuroimaging

Do You Act Before You Think or Think Before You Act?

Summary: Even though people engage in seemingly impulsive decision-making, they may actually be predisposed to curtail it. Source: UCR Will you read this news release now or decide to read it later? “Precrastination,” defined as the tendency to expend extra effort to get things done as soon as possible, could ...

View more: Do You Act Before You Think or Think Before You Act?

A Possible Brain Mechanisms Behind COVID-19 Delirium

Summary: Brain cells exposed to blood taken from COVID-19 patients with delirium showed a decrease in neurogenesis and an increase in cell death. The findings also indicate a key role for cytokines produced in the immune system during infection and shed light on the molecular mechanisms of COVID-19-related delirium. Source: ...

View more: A Possible Brain Mechanisms Behind COVID-19 Delirium

Researchers develop new tool for targeted cell control

Credit: Pixabay/CC0 Public Domain Thanks to new RNA vaccines, we humans have been able to protect ourselves incredibly quickly from new viruses like SARS-CoV-2, the virus that causes COVID-19. These vaccines insert a piece of ephemeral genetic material into the body’s cells, which then read its code and churn ...

View more: Researchers develop new tool for targeted cell control

Researchers pioneer nanoprinting electrodes for customized treatments of neurological disorders

Researchers used Aerosol Jet 3D printing to produce a new type of microelectrode array that solves the major design barriers of other brain computer interface arrays. Credit: College of Engineering Carnegie Mellon University Carnegie Mellon University researchers have pioneered the CMU Array—a new type of microelectrode array for brain ...

View more: Researchers pioneer nanoprinting electrodes for customized treatments of neurological disorders

Study shows challenge of promoting citizen science to help prevent disasters caused by flooding

Waterproofing Data team interviewing residents of M’Boi Mirim district in São Paulo, 2019. Credit: Waterproofing Data An application developed in collaboration with vulnerable communities to transmit early warnings of the risk of floods and environmental disasters has become public policy in Brazil, and the methodology is being replicated for ...

View more: Study shows challenge of promoting citizen science to help prevent disasters caused by flooding

Crew Dragon launches safely, carrying first Russian from US soil in 20 years

Click! 2022 Nobel goes to chemistry made simple and reliable

A Nose to Diagnose: Improving Parkinson’s Diagnosis

Multiple Health Benefits of Chocolate, Red Wine, and Apples When Consumed in the Right Amounts

RNA origami enables applications in synthetic biology

Empathizing with the opposition may make you more politically persuasive

Birdsong reveals rare hybrid coupling 10 million years in the making

Apple Seeds Eighth Public Beta of macOS 13 Ventura

Do I need to refrigerate pumpkin pie? Food safety info for pumpkin season

SpaceX Brings Russian Cosmonaut, Other NASA Astronauts to International Space Station

Catch-and-release fishing may cause temperature spikes in sharks

Thanks to modern science and one artist's vision, plants can now wield swords

OTHER TECH NEWS

Top Car News Car News