Raiders of the lost HARK: a reproducible inference framework for big data science

Prosperi, Mattia, Bian, Jiang, Buchan, Iain E ORCID: 0000-0003-3392-1650, Koopman, James S, Sperrin, Matthew and Wang, Mo (2019) Raiders of the lost HARK: a reproducible inference framework for big data science. PALGRAVE COMMUNICATIONS, 5 (1). 125-.

Access the full-text of this item by clicking on the Open Access link.

Official URL: http://dx.doi.org/10.1057/s41599-019-0340-8

Abstract

<jats:title>Abstract</jats:title><jats:p>Hypothesizing after the results are known (HARK) has been disparaged as data dredging, and safeguards including hypothesis preregistration and statistically rigorous oversight have been recommended. Despite potential drawbacks, HARK has deepened thinking about complex causal processes. Some of the HARK precautions can conflict with the modern reality of researchers’ obligations to use big, ‘organic’ data sources—from high-throughput genomics to social media streams. We here propose a HARK-solid, reproducible inference framework suitable for big data, based on models that represent formalization of hypotheses. Reproducibility is attained by employing two levels of model validation: internal (relative to data collated around hypotheses) and external (independent to the hypotheses used to generate data or to the data used to generate hypotheses). With a model-centered paradigm, the reproducibility focus changes from the ability of others to reproduce both data and specific inferences from a study to the ability to evaluate models as representation of reality. Validation underpins ‘natural selection’ in a knowledge base maintained by the scientific community. The community itself is thereby supported to be more productive in generating and critically evaluating theories that integrate wider, complex systems.</jats:p>

Item Type:	Article
Uncontrolled Keywords:	Networking and Information Technology R&D (NITRD), Generic health relevance
Depositing User:	Symplectic Admin
Date Deposited:	02 Dec 2019 13:28
Last Modified:	15 Mar 2024 15:22
DOI:	10.1057/s41599-019-0340-8
Open Access URL:	https://doi.org/10.1057/s41599-019-0340-8
Related URLs:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3064439