If you read the headlines, medical scientists are amazing. It seems every day, they discover a new cure for cancer or the genetic basis of some prominent disease. With all the cures, keys, breakthroughs and discoveries, it’s a wonder anyone still gets sick.
Of course, readers soon learn the truth: a lot of science reporting is sensationalized nonsense. Hyping science a vicious cycle. Scientists work hard, get results, and publish. Press officers try to publicize these results, then journalists build off the press releases, and before you know it, your grandmother is wearing a tin foil hat. This is predictably followed by angry scientists and science writers with their rolled up newspapers swatting the noses of the “churnalists” for their bad reporting. People like Ed Yong and I feel forced to don our latex gloves and clean up the crap left on the carpet, all the while sternly saying “Bad, journalist. BAD!”.
But are journalists, as a whole, really that bad at their jobs? No, actually, says a new paper published today in PLoS Medicine. It’s not all the writers’ fault: when they examined the language used in press releases and the studies themselves, instead, it was the scientists and their press offices that were largely to blame.
A team of French scientists led by Isabelle Boutron from the Université Paris Descartes sought to get to the bottom of why medical news is so over-spun. They examined the language in clinical trials along with their associated press releases and news reports for spin—defined as specific reporting strategies emphasizing the beneficial effect of the experimental treatment—to see exactly where the hype comes from.
As expected, they found that the media’s portrayal of results was often sensationalistic. More than half of the news items they examined contained spin. But, while the researchers found a lot of over-reporting, they concluded that most of it was “probably related to the presence of ‘‘spin’’ in conclusions of the scientific article’s abstract.”
It turned out that 47% of the press releases contained spin. Even more importantly, of the studies they examined, 40% of the study abstracts or conclusions did, too. When the study itself didn’t contain spin to begin with, only 17% of the news items were sensationalistic, and of those, 3/4 got their hype from the press release.
In the journal articles themselves, they found that authors spun their own results a variety of ways. Most didn’t acknowledge that their results were not significant or chose to focus on smaller, significant findings instead of overall non-significant ones in their abstracts and conclusions, though some contained outright inappropriate interpretations of their data.
The press releases often built off of the spin in the studies. Of the press releases that contained spin, 93% were from studies that had spin in their abstracts. In fact, spin present in the study was the only significant factor associated with spin in the press release. A whopping 31% of press releases misinterpreted the scientists’ findings, with the vast majority conflating the benefits of the study’s tested treatment.
It’s not news that press releases are skewed. Previous research found that most press releases left out important caveats on safety or applicability of the research, and many flat out exaggerated the importance of results. “Our study adds to these results showing that ‘‘spin’’ in press releases and the news is related to the presence of ‘‘spin’’ in the published article,” say the authors. In other words – the root of the problem lies in how we write up research results in the first place.
The authors were sure to note that while their results are striking, their study has limitations. They ended up with only 41 trials paired with press releases and news articles—a small sample size with which to examine the whole of medical news reporting. They also focused solely on randomized controlled trials, a small subset of all medical research. Still, they feel that their results require further investigation, and that the burden of ensuring scientific rigor in reporting falls on the peer review system. “Reviewers and editors of published articles have an important role to play in the dissemination of research findings and should be particularly aware of the need to ensure that the conclusions reported are an appropriate reflection of the trial findings and do not overinterpret or misinterpret the results.”
All of this is not to say journalists are entirely innocent. Good journalism requires that you look beyond the press release to get at the heart of the study, and great science journalists know to take anything that comes out of a press office with a grain of salt. They read the study itself, and talk to not only the scientists who wrote the study but also other scientists in the field to really understand the importance of the research involved. Churnalism is definitely a problem that needs to be addressed alongside concerns of scientist bias and hyped press releases. Researchers, press officers and journalists all need to take responsibility for accurate and informative science communication.
Citation: Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, et al. (2012) Misrepresentation of Randomized Controlled Trials in Press Releases and News Coverage: A Cohort Study. PLoS Med 9(9): e1001308. DOI: 10.1371/journal.pmed.1001308.t004
