The COVID-19 pandemic has led to widespread questioning of federal public health policies and the scientific evidence supporting them. This is a result, at least in part, of the lack of data transparency in federal health journals shaping public policy. It is high time that these journals adopt the standards of transparency used by the leading journals in the private sector, or continue to lose public and scholarly trust.
The federal Department of Health and Human Services (HHS) creates health policy using peer-reviewed research, including publications in four scientific health journals it oversees and manages: Emerging Infectious Diseases, Mortality and Morbidity Weekly Report, Environmental Health Perspectives, and recently federalized Journal of Health and Pollution.
None of these journals adhere to the most modest transparency policies set out by Oxford University Press – simply encouraging authors, when ethically possible, to publicly release all data underlying a published article. The strongest transparency policy identified by Oxford requires “all authors, where ethically possible, to publicly release all data underlying any published paper as a condition of publication” and subjects the data to peer review.
This lack of attention to independent reproducibility is striking, since a defining characteristic of science—which is perhaps necessary for public trust—is that independent researchers using the same analytical methods reach the same conclusion. Additionally, a 2019 study on reproducibility and reproducibility by the National Academies of Sciences concluded that “reproducibility is strongly related to transparency; the research data and code must be available for others to reproduce and confirm the results.”
Policies requiring access to data and computer code as a condition of publication exist in highly regarded journals affiliated with Science, Nature, and the American Economic Association, as well as in the Proceedings of the National Academies of Science (PNAS), Environmental Science & Technology, and so called Science, Nature, and PNAS are multidisciplinary journals, so their procedures, which are designed to protect people’s privacy and recognize the use of private data, are designed to be applicable to a wide variety of research areas.
At least one federal journal has already adopted better data access policies. The federal Journal of Fish and Wildlife Management now requires public disclosure of the underlying data.
The independent reproducibility of HHS research is not just an academic concern. In 2020, the US Government Accountability Office investigated HHS infectious disease modeling, finding that guidance and policy decisions from the Centers for Disease Control and Prevention (CDC) did not address model or code reproducibility—a shortcoming that may have compromised the reliability of CDC research on COVID-19. It recommended that CDC establish guidelines to ensure full reproducibility of its research, such as sharing with the public all admissible and relevant information necessary to reproduce research results, including, but not limited to, model code.
HHS, or perhaps the White House or Congress, should require all federal health journals to adopt policies for public access to researchers’ data and the code of leading scientific journals. If these journals are unable to adopt such policies, it is not clear why they might deserve taxpayer support.
Randall LutherPh.D., is a senior fellow at Manhattan Institute, where his research focuses on pharmaceutical markets and policy, medical innovation and regulation. He was the Senior Scientific and Regulatory Advisor at the Office of the Commissioner at the US Food and Drug Administration from 2017 to 2020.