Did you hear the news today? Science has finally proven that men are bigger idiots than women.
Men: Hold on and don’t get too defensive just yet. Women: Don’t gloat, or we may turn out to be the idiots here. Because there are a few problems with this news that we should talk about.
This much is true: One of the world’s leading scientific journals, the British Medical Journal (BMJ), published this study, positing the existence of “Male Idiot Theory” and providing evidence for that theory based on real information about avoidable, stupid deaths.
But this study, “The Darwin Awards: sex differences in idiotic behavior,” was part of BMJ’s annual Christmas issue which includes a section of tongue-in-cheek science—in which scientists show their lighter side by analyzing real data to address silly questions, often in silly ways.
If you actually read the study, or stop and think about it seriously, there are obvious limitations and questions. The scientists studied only a very small sample of idiotic behavior, of a totally unscientific award for people who died in bizarre and preventable ways (when someone else was around to document and report on the details afterward). If this were any normal study in a normal issue of a top-tier journal, the editor would reject a paper for this lack of rigor in the data and sampling methods. And then there are other questions, such as: Is dying in an idiotic way really a universal measure of idiocy, or just its most irreversible case?
Only some of the media reports on the study have pointed out the joke. It’s hard to put too much blame on the believers, since BMJ actually issued a tongue-in-cheek press release that doesn’t explicitly point it out. Is the dissemination of this fake news making idiots out of all of us? (Should there be a Literally Unbelievable for science?)
This happens every year, according to Lawrence Souder, PhD, an associate teaching professor in Drexel University’s College of Arts and Sciences, who studies science communication, rhetoric and ethics. These credulous accounts of BMJ’s Christmas studies and other instances of joke science aren’t limited to just the popular news media; Souder and a former student, Maryam Ronagh, have documented that credulous citations of what they call “ironic science” subsequently appear in future scientific research on related topics. Their case study analysis of a study from a past BMJ Christmas issue has been accepted for publication and will appear in a forthcoming issue of the journal Science and Engineering Ethics.
Ronagh and Souder examined responses to and citations of a study showing that intercessory prayer—praying on behalf of a patient hospitalized with a serious infection—had an impact on patient outcomes. But this particular study was retroactive, meaning that all of the patients were already dead prior to the start of the study. It was published in BMJ’s Christmas issue in 2001. It’s a classic case of irony, the practice of saying the opposite of one’s intended meaning for humorous or emphatic expression. The author of that study actually did compare “results” of two patient groups who randomly either received or did not receive retroactive prayer, but he later clarified in a letter that his intent was not really to test the effects of prayer or to mock those who believe in it—it was to make a point that prayer and other matters of faith, by definition, shouldn’t be subject to empiricism.
The author of that paper needed to clarify his intent because he was so widely misunderstood. Ronagh and Souder found that many of the responses and more than 60 subsequent citations of the retroactive prayer paper overlooked the illogic of praying to extend the lives of patients after they had already died; some pointed out other problems, such as that the dead patients did not provide informed consent to be research subjects, or that the religious denomination of the praying volunteers was not noted. Some papers did criticize the study for its retroactive design or pointed out that it was a hoax or parody. And some were confused by the incongruity of the evident problems. (“Those who expressed confusion over his sincerity found that the very context of science communication had become unknowable—a veritable Alice in Wonderland scenario where lab coats have replaced top hats,” Ronagh and Souder write.)
But a number of scientific papers cited the prayer study admiringly or otherwise seriously—including the Cochrane Review, a well-regarded meta analysis that vets and weighs the evidence from multiple scientific studies to determine the scientific consensus on a medical question.
This misinterpretation of ironic science is bad for science, Ronagh and Souder argue.
“It is troublesome because it’s a waste of the time of the researchers,” said Ronagh, who now teaches in the publication management master’s degree program at Drexel. “When I’m going to Web of Science [database of peer-reviewed published research] as a researcher, I want to know that what I’m searching is valid. People don’t have time to reevaluate something that they’re finding in a research database.”
This challenge mirrors the difficulties caused for scientists and scientific publishers when research papers are retracted.
“Frankly, ironic science is not that different than retracted research,” Souder said. “You don’t want people to take retracted research as legitimate, and you don’t want them to take ironic research as legitimate, either.”
So does that mean scientists are doomed to always be serious?
“I’d hate to see humor go away from the scientific record,” Souder said. “There should be some way to allow ironic science to be part of the record, but to have a kind of red flag attached to it as well. There isn’t a formal mechanism for doing that.”
Souder and Ronagh recommend flagging ironic and other joke research as such upon publication and keeping it out of databases that scientists use to search previous literature.
“We want some way of identifying on the record, in the archive, that this is not to be taken seriously, when it’s placed along with the zillions of other articles that are actually serious,” Souder said.
He and Ronagh point out that readers can’t always assume that the content of a scientific article is certain, proven truth—the nature of science itself is provisional—but they should have confidence in the truthful intent of its authors. Otherwise, the joke’s on all of us.
