This is a preview of subscription content
Subscription info for Chinese customers
We have a dedicated website for our Chinese customers. Please go to naturechina.com to subscribe to this journal.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
No data have been generated as part of this manuscript.
McKinney, S. M. et al. International evaluation of an AI system for breast cancer screening. Nature 577, 89–94 (2020).
Bluemke, D. A. et al. Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers—from the Radiology editorial board. Radiology 293, 315–316 (2019).
Gundersen, O. E., Gil, Y. & Aha, D. W. On reproducible AI: towards reproducible research, open science, and digital scholarship in AI publications. AI Mag. 39, 56–68 (2018).
Crane, M. Questionable answers in question answering research: reproducibility and variability of published results. Trans. Assoc. Comput. Linguist. 6, 241–252 (2018).
Sculley, D. et al. in Advances in Neural Information Processing Systems 28 (eds Cortes, C. et al.) 2503–2511 (Curran Associates, Inc., 2015).
Stodden, V. et al. Enhancing reproducibility for computational methods. Science 354, 1240–1241 (2016).
Hutson, M. Artificial intelligence faces reproducibility crisis. Science 359, 725–726 (2018).
Bzdok, D. & Ioannidis, J. P. A. Exploration, inference, and prediction in neuroscience and biomedicine. Trends Neurosci. 42, 251–262 (2019).
Gundersen, O. E. & Kjensmo, S. State of the art: Reproducibility in artificial intelligence. In Thirty-second AAAI Conference on Artificial Intelligence (AAAI-18) 1644–1651 (2018).
Shorten, C. & Khoshgoftaar, T. M. A survey on Image Data Augmentation for Deep Learning. J. Big Data 6, 60 (2019).
Lee, R. S. et al. A curated mammography data set for use in computer-aided detection and diagnosis research. Sci. Data 4, 170177 (2017).
Wallach, J. D., Boyack, K. W. & Ioannidis, J. P. A. Reproducible research practices, transparency, and open access data in the biomedical literature, 2015-2017. PLoS Biol. 16, e2006930 (2018).
Amann, R. I. et al. Toward unrestricted use of public genomic data. Science 363, 350–352 (2019).
Carlson, B. Putting oncology patients at risk. Biotechnol. Healthc. 9, 17–21 (2012).
We thank S. McKinney and colleagues for their prompt and open communication regarding the materials and methods of their study. This work was supported in part by the National Cancer Institute (R01 CA237170).
A.H. is a shareholder of and receives consulting fees from Altis Labs. M.M.H. received a GPU Grant from Nvidia. H.J.W.L.A. is a shareholder of and receives consulting fees from Onc.AI. B.H.K. is a scientific advisor for Altis Labs. C.M. holds an equity position in Bridge7Oncology and receives royalties from RaySearch Laboratories. A.K. is on the SAB of ImmuneAI Inc, a consultant for Biogen Inc., a scientific co-founder of RavelBio Inc. and a shareholder of Freenome Inc. G.A.A., F.K., L.W., B.W., C.S.G., J.T.L., W.H., A.B., J.P., R.T., T.H., J.P.A.I. and J.Q. declare no other competing interests related to the manuscript.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Haibe-Kains, B., Adam, G.A., Hosny, A. et al. Transparency and reproducibility in artificial intelligence. Nature 586, E14–E16 (2020). https://doi.org/10.1038/s41586-020-2766-y
The reporting quality of natural language processing studies: systematic review of studies of radiology reports
BMC Medical Imaging (2021)
Communications Biology (2021)
Nature Reviews Cancer (2021)
npj Precision Oncology (2021)
Nature Reviews Cardiology (2021)