Psychology’s replication crisis inspires ecologists

Psychology’s replication crisis inspires ecologists to push for more reliable research

Cathleen O’Grady

Ecology suffers from many of the same underlying problems as psychology. Surveys of the ecology literature have found that small sample sizes are common, often driven by high cost or limited access to a species or other study system. In landscape ecology, each landscape is unique, meaning the sample size is one, Wiersma says. “There is one Yellowstone park,” she says. “There’s one Lake District.” Small samples lead to erratic results that sometimes miss the effects researchers are looking for and other times hit on noise that looks like a real signal.

Worsening those problems are “questionable research practices,” says Fiona Fidler, a metascientist at the University of Melbourne. In a 2018 study published in PLOS ONE, Parker, Fidler, and colleagues reported on a survey of more than 800 ecologists and evolutionary biologists. About half of the respondents said they sometimes presented unexpected findings as if they confirmed a hypothesis they’d had all along, and about two-thirds said they sometimes reported only significant results, leaving out negative ones. Together, these forces mean a literature overflowing with potentially dubious results, Parker says. It’s a “house of cards.”

But unlike psychology, in which researchers have tried to replicate famous studies and failed in about half the cases, ecology has no smoking gun. A 2019 PeerJ study found only 11 replication studies among nearly 40,000 ecology and evolution biology papers—and only four of these 11 studies managed to replicate the original finding. It’s hard to replicate ecology studies, Parker says, because it often entails expensive and difficult data gathering in remote places or over long time frames. And ecosystems are so complex that any number of variables could affect the outcome of a repeat experiment—like the age of the organisms in the study, the temperatures at the time, or the presence or absence of pollutants. “No man can step into the same river twice because it’s not the same man and it’s not the same river,” says Phillip Williamson, an ecologist at the University of East Anglia who has criticized a high-profile effort to replicate ocean acidification research.

Yet Williamson doesn’t think ecology as a whole is at risk just because some experiments fail to replicate. “Biology isn’t physics,” he says. “I think that the consensus of science gets there eventually.” Parker takes a harder line. “If we don’t expect anything to replicate, why do we bother doing any of this?” he asks.