Indirect evidence of reporting biases was found in a survey of medical research studies

Research output: Contribution to journalArticle (Academic Journal)peer-review

3 Citations (Scopus)
247 Downloads (Pure)

Abstract

Objectives

To explore indirect evidence of reporting biases by examining the distribution of P-values reported in published medical articles and to compare P-values distributions across different contexts.

Study Design and Setting

We selected a random sample (N = 1,500) of articles published in PubMed in March 2014. We extracted information on study type, design, medical discipline, and P-values for the first reported outcome and primary outcome (if specified) from each article. We plotted the P-values transformed to the z-score scale and used caliper tests to investigate threshold effects.

Results

Out of the 1,500 randomly selected records, 758 (50.5%) were included. We retrieved or calculated 758 P-values for first reported outcomes and 389 for primary outcomes (specified in only 51% of included studies). The first reported and the primary outcome differed in 28% (110/389) of the included studies. The distributions of P-values for first reported outcomes and primary outcomes showed a notable discontinuity at the common thresholds of statistical significance (P-value = 0.05 and P-value = 0.01). We also found marked discontinuities in the distributions of z-scores across various medical disciplines, study designs, and types.

Conclusion

Reporting biases are still common in medical research. We discuss their implications, strategies to detect them, and recommended practices to avoid them.
Original languageEnglish
Pages (from-to)57-64
Number of pages8
JournalJournal of Clinical Epidemiology
Volume83
Early online date11 Jan 2017
DOIs
Publication statusPublished - Mar 2017

Keywords

  • bias
  • p-curve
  • p-hacking
  • methodology
  • reporting bias
  • publication bias

Fingerprint Dive into the research topics of 'Indirect evidence of reporting biases was found in a survey of medical research studies'. Together they form a unique fingerprint.

Cite this