Chapter 2
Experimental research
Problems and opportunities in the big-data era
Experimental research in psychology, psycholinguistics or medicine provides quantitative and therefore seemingly conclusive and trustworthy evidence. However, it has been convincingly shown that most research findings are actually false. This has hardly influenced the dominant scientific evaluation system which reflects a continued trust in the unbiasedness of data by a strong reliance on simple quantifications of scientific quality and productivity, such as number of publications and number of citations. This state of affairs is remarkable in the light of a long history of strong criticism of commonly used inference methods and scientific evaluation systems, which is now backed by large-scale research projects directly questioning the reproducibility of scientific findings. This way, the large amounts of data – “big-data” – have helped to uncover some of these problematic issues, but also provided a more open attitude towards data and code sharing. In addition, novel analytic frameworks may help to better integrate empirical data with computational models.
Article outline
- 1.Introduction
- 2.The questionable empirical toolbox
- 2.1Bias
- 2.2Null hypothesis testing
- 2.3Theory testing
- 3.Scientific publications and evaluation
- 3.1The economy of the publication and evaluation systems
- 3.2Alternatives for the evaluation system
- 4.More data, more problems?
-
Notes
-
References