Perspectives on scientific error

D van Ravenzwaaij; M Bakker; R Heesen; F Romero; N van Dongen; S Crüwell; S M Field; L Held; M R Munafò; M M Pittelkow; L Tiokhin; V A Traag; O R van den Akker; A E van 't Veer; E J Wagenmakers

doi:10.1098/rsos.230448

Perspectives on scientific error

D van Ravenzwaaij^*
, M Bakker
, R Heesen
, F Romero
, N van Dongen
, S Crüwell
, S M Field
, L Held
, M R Munafò
, M M Pittelkow
, L Tiokhin
, V A Traag
, O R van den Akker
, A E van 't Veer
, E J Wagenmakers

^*Corresponding author for this work

Research output: Contribution to journal › Review article (Academic Journal) › peer-review

14 Citations (Scopus)

Abstract

Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.

Original language	English
Article number	230448
Number of pages	15
Journal	Royal Society Open Science
Volume	10
Issue number	7
DOIs	https://doi.org/10.1098/rsos.230448
Publication status	Published - 19 Jul 2023

Bibliographical note

Funding Information:
After our fictitious researcher Riley has decided to apply for a grant to fund the extension study on y, Riley is faced with the task of having to write a grant proposal that maximizes the chances of getting funded. Writing a competitive grant proposal involves elements related to the quality of the scientific proposal and the quality of the candidate. Both need to be communicated to a panel of experts who are typically from different fields than Riley. Humans are imperfect at processing information, and the panel may employ some heuristics in their decision-making. The quality of the scientific proposal may be rated higher if the topic is familiar (the mere exposure effect []) and the quality of the candidate may be rated higher if they have a long list of publications, are highly cited, or publish in journals with high impact factors. Such indicators may be used as heuristics, especially for people unfamiliar with a field. Riley argues for the quality of the scientific proposal through the selection of y as a study topic (recall that y has received a lot of media attention) and argues for the quality of track record through a CV with many publications that were well cited, a result of Riley having prioritized publishing studies on low-hanging fruit combined with liberal self-citations in the past.

Publisher Copyright:
© 2023 The Authors.

Research Groups and Themes

TARG

Access to Document

10.1098/rsos.230448Licence: CC BY

Handle.net

Persistent link

Cite this

@article{2a136b79dd944323a9e3f9b512f52529,

title = "Perspectives on scientific error",

abstract = "Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings. ",

author = "\{van Ravenzwaaij\}, D and M Bakker and R Heesen and F Romero and \{van Dongen\}, N and S Cr{\"u}well and Field, \{S M\} and L Held and Munaf{\`o}, \{M R\} and Pittelkow, \{M M\} and L Tiokhin and Traag, \{V A\} and \{van den Akker\}, \{O R\} and \{van 't Veer\}, \{A E\} and Wagenmakers, \{E J\}",

note = "Funding Information: After our fictitious researcher Riley has decided to apply for a grant to fund the extension study on y, Riley is faced with the task of having to write a grant proposal that maximizes the chances of getting funded. Writing a competitive grant proposal involves elements related to the quality of the scientific proposal and the quality of the candidate. Both need to be communicated to a panel of experts who are typically from different fields than Riley. Humans are imperfect at processing information, and the panel may employ some heuristics in their decision-making. The quality of the scientific proposal may be rated higher if the topic is familiar (the mere exposure effect []) and the quality of the candidate may be rated higher if they have a long list of publications, are highly cited, or publish in journals with high impact factors. Such indicators may be used as heuristics, especially for people unfamiliar with a field. Riley argues for the quality of the scientific proposal through the selection of y as a study topic (recall that y has received a lot of media attention) and argues for the quality of track record through a CV with many publications that were well cited, a result of Riley having prioritized publishing studies on low-hanging fruit combined with liberal self-citations in the past. Publisher Copyright: {\textcopyright} 2023 The Authors.",

year = "2023",

month = jul,

day = "19",

doi = "10.1098/rsos.230448",

language = "English",

volume = "10",

journal = "Royal Society Open Science",

issn = "2054-5703",

publisher = "The Royal Society",

number = "7",

}

TY - JOUR

T1 - Perspectives on scientific error

AU - van Ravenzwaaij, D

AU - Bakker, M

AU - Heesen, R

AU - Romero, F

AU - van Dongen, N

AU - Crüwell, S

AU - Field, S M

AU - Held, L

AU - Munafò, M R

AU - Pittelkow, M M

AU - Tiokhin, L

AU - Traag, V A

AU - van den Akker, O R

AU - van 't Veer, A E

AU - Wagenmakers, E J

N1 - Funding Information: After our fictitious researcher Riley has decided to apply for a grant to fund the extension study on y, Riley is faced with the task of having to write a grant proposal that maximizes the chances of getting funded. Writing a competitive grant proposal involves elements related to the quality of the scientific proposal and the quality of the candidate. Both need to be communicated to a panel of experts who are typically from different fields than Riley. Humans are imperfect at processing information, and the panel may employ some heuristics in their decision-making. The quality of the scientific proposal may be rated higher if the topic is familiar (the mere exposure effect []) and the quality of the candidate may be rated higher if they have a long list of publications, are highly cited, or publish in journals with high impact factors. Such indicators may be used as heuristics, especially for people unfamiliar with a field. Riley argues for the quality of the scientific proposal through the selection of y as a study topic (recall that y has received a lot of media attention) and argues for the quality of track record through a CV with many publications that were well cited, a result of Riley having prioritized publishing studies on low-hanging fruit combined with liberal self-citations in the past. Publisher Copyright: © 2023 The Authors.

PY - 2023/7/19

Y1 - 2023/7/19

N2 - Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.

AB - Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.

U2 - 10.1098/rsos.230448

DO - 10.1098/rsos.230448

M3 - Review article (Academic Journal)

C2 - 37476516

SN - 2054-5703

VL - 10

JO - Royal Society Open Science

JF - Royal Society Open Science

IS - 7

M1 - 230448

ER -

Perspectives on scientific error

Abstract

Bibliographical note

Research Groups and Themes

Access to Document

Handle.net

Fingerprint

Cite this