Mask or No Mask? Coffee or No Coffee? Magdalena Luca on Finding Scientific Information You Can Trust

If you make a practice of listening to the news, or leafing through a morning newspaper, then you already know that coffee is terrific for you… unless it’s actually terrible. No, wait: coffee is full of antioxidants, so you should be okay. Never mind—a new study says should all be drinking green tea instead. Or is that red wine? And are eggs good this week, or bad?

Sifting through conflicting health information can be overwhelming on the best of days; but during a pandemic, the pressure to sort good scientific advice from bad becomes even more heightened.

Thankfully, Magdalena Luca, PhD, Professor of Math and Statistics at MCPHS University, is here to help you make sense of this sea of conflicting information. Her Zoom lecture entitled “Mask or no mask? Coffee or no coffee? Navigating the complicated maze of scientific information” was the third installment of MCPHS’s faculty webinar series, “Navigating the New Normal,” and focused specifically on the analytical skills needed to properly evaluate scientific reports.

“It takes a very long time to be able to navigate scientific info properly, and even when we are knowledgeable, it can be very complicated,” said Luca, whose scientific career spans over three decades. “The three types of scientific information are scholarly, substantial, and popular.” Scholarly writing is produced by experts, with an assumed audience of fellow experts, and is subject to a rigorous process of peer review. Substantive scientific writing is heavily fact-checked but is written with a lay audience in mind: for example, an article in Scientific American or Nature. Finally, there is popular writing; this can include broad journalism such as a daily newspaper or television report and is subject to a wide range of accuracy standards, depending on the source. “You must always assess several different criteria to evaluate a source,” said Luca, “including format, the currency (i.e. how recently it was published), relevance, authors, accuracy, and purpose.”

One of the key parts of ensuring quality in scholarly writing is peer review. The peer review process defines how scholarly works are created and evaluated. Often taking years, the process entails painstaking revision based on notes from a panel of experts. “Peer review is the gold standard for how scholarly works are done,” said Luca. “But since time is of the essence with the pandemic, many articles have been published while still awaiting peer review.”

How are these new, often not-yet-peer-reviewed articles to be evaluated? Luca urges caution. When evaluating a new study, the professor always starts by carefully examining the study’s funding, sources, and sponsors to discern possible conflicts of interest. “Then, I look at the sample size, length of the study, and finally I ask if confounding factors are addressed,” said Luca. Confounding factors are the complex interplay of conditions which complicate the results of any study: for example, in a study of a cholesterol drug, diet and exercise would represent confounding factors. “In the last 10 years, the statistical methods that address confounding factors have dramatically improved, so you often have to look at older studies with a bit of skepticism about possible confounding factors,” reported Luca.

Luca urges caution for good reason. A famous example of a highly flawed paper whose later retraction did little to halt the spread of dangerous misinformation about its subject—vaccines—is the Wakefield paper. “The Wakefield paper was published in the Lancet; a top medical journal, in 1998,” said Luca. “Problems with its funding don’t jump out immediately; but there are only 12 children in the sample size.” The study claimed a connection between the measles, mumps, and rubella (MMR) vaccine and autism; at the time, multiple scientists noted the small sample size and uncontrolled design of the study, along with its highly vague and speculative conclusions. Subsequently, hundreds of studies were conducted (including one published in the New England Journal of Medicine following 600,000 children in Denmark) that found no connection whatsoever between vaccines and autism; unfortunately, the damage had already been done. The Wakefield paper is considered the parent of the modern anti-vaccination movement and has persisted in public circulation despite an investigation revealing enormous conflicts of interest in its funding, deliberate fraud, and profound ethical violations. (In 2010, the author his UK medical license.)

With popular articles, Luca cautioned that it is always crucial to pay close attention to the original scientific study on which the article is based. “In good newspapers, you will always find a link to the original study,” she noted, adding that it is always important to consult the original study to ensure that the subsequent article does not oversimplify or misread its findings. Looking at studies of SARS-CoV-2, Luca noted that the available information about the virus is evolving quickly. “We know more about the modes of transmission of SARS-CoV-2 than we knew last month; in another month, we may know even more,” she said.

Concluding her talk on an upbeat note, Luca expressed hope that “as we all get older, the glass of knowledge fills. Ideally, we should gain more knowledge of how to evaluate sources and navigate the maze of information. So: wear a mask, wash your hands, and watch your distance.” And, in a final ruling on decades of scientific back-and-forth, Luca added that we should go ahead and have the cup of coffee. Good to know.