Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

A.P. Christie; D. Abecasis; M. Adjeroud; J.C. Alonso; T. Amano; A. Anton; B.P. Baldigo; R. Barrientos; J.E. Bicknell; D.A. Buhl; J. Cebrian; R.S. Ceia; L. Cibils-Martina; S. Clarke; J. Claudet; M.D. Craig; D. Davoult; A. De Backer; Mary K. Donovan; T.D. Eddy; Filipe M. França; Jonathan P.A. Gardner; Bradley P. Harris; A. Huusko; Ian L. Jones; B.P. Kelaher; J.S. Kotiaho; A. López-Baucells; H.L. Major; A. Mäki-Petäys; B. Martín; Carlos A. Martín; Philip A. Martin; D. Mateos-Molina; R.A. McConnaughey; M. Meroni; C.F.J. Meyer; Kade Mills; M. Montefalcone; N. Noreika; C. Palacín; Anjali Pande; C.R. Pitcher; C. Ponce; M. Rinella; R. Rocha; M.C. Ruiz-Delgado; J.J. Schmitter-Soto; J.A. Shaffer; Shailesh Sharma; A.A. Sher; D. Stagnol; T.R. Stanley; K.D.E. Stokesbury; A. Torres; O. Tully; T. Vehanen; C. Watts; Qingyuan Zhao; W.J. Sutherland

doi:10.1038/s41467-020-20142-y

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

A.P. Christie, D. Abecasis, M. Adjeroud, J.C. Alonso, T. Amano, A. Anton, B.P. Baldigo, R. Barrientos, J.E. Bicknell, D.A. Buhl, J. Cebrian, R.S. Ceia, L. Cibils-Martina, S. Clarke, J. Claudet, M.D. Craig, D. Davoult, A. De Backer, Mary K. Donovan, T.D. EddyFilipe M. França, Jonathan P.A. Gardner, Bradley P. Harris, A. Huusko, Ian L. Jones, B.P. Kelaher, J.S. Kotiaho, A. López-Baucells, H.L. Major, A. Mäki-Petäys, B. Martín, Carlos A. Martín, Philip A. Martin, D. Mateos-Molina, R.A. McConnaughey, M. Meroni, C.F.J. Meyer, Kade Mills, M. Montefalcone, N. Noreika, C. Palacín, Anjali Pande, C.R. Pitcher, C. Ponce, M. Rinella, R. Rocha, M.C. Ruiz-Delgado, J.J. Schmitter-Soto, J.A. Shaffer, Shailesh Sharma, A.A. Sher, D. Stagnol, T.R. Stanley, K.D.E. Stokesbury, A. Torres, O. Tully, T. Vehanen, C. Watts, Qingyuan Zhao, W.J. Sutherland

Research output: Contribution to journal › Article (Academic Journal) › peer-review

65 Citations (Scopus)

Abstract

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

Original language	English
Journal	Nature Communications
DOIs	https://doi.org/10.1038/s41467-020-20142-y
Publication status	Published - 11 Dec 2020

Access to Document

10.1038/s41467-020-20142-y

Handle.net

Persistent link

Cite this

Christie, A. P., Abecasis, D., Adjeroud, M., Alonso, J. C., Amano, T., Anton, A., Baldigo, B. P., Barrientos, R., Bicknell, J. E., Buhl, D. A., Cebrian, J., Ceia, R. S., Cibils-Martina, L., Clarke, S., Claudet, J., Craig, M. D., Davoult, D., Backer, A. D., K. Donovan, M., ... Sutherland, W. J. (2020). Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences. Nature Communications. https://doi.org/10.1038/s41467-020-20142-y

@article{3748921c44ed49ea995b966dd47c335e,

title = "Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences",

abstract = "Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs. ",

author = "A.P. Christie and D. Abecasis and M. Adjeroud and J.C. Alonso and T. Amano and A. Anton and B.P. Baldigo and R. Barrientos and J.E. Bicknell and D.A. Buhl and J. Cebrian and R.S. Ceia and L. Cibils-Martina and S. Clarke and J. Claudet and M.D. Craig and D. Davoult and Backer, {A. De} and {K. Donovan}, Mary and T.D. Eddy and Fran{\c c}a, {Filipe M.} and {P.A. Gardner}, Jonathan and {P. Harris}, Bradley and A. Huusko and {L. Jones}, Ian and B.P. Kelaher and J.S. Kotiaho and A. L{\'o}pez-Baucells and H.L. Major and A. M{\"a}ki-Pet{\"a}ys and B. Mart{\'i}n and {A. Mart{\'i}n}, Carlos and {A. Martin}, Philip and D. Mateos-Molina and R.A. McConnaughey and M. Meroni and C.F.J. Meyer and Kade Mills and M. Montefalcone and N. Noreika and C. Palac{\'i}n and Anjali Pande and C.R. Pitcher and C. Ponce and M. Rinella and R. Rocha and M.C. Ruiz-Delgado and J.J. Schmitter-Soto and J.A. Shaffer and Shailesh Sharma and A.A. Sher and D. Stagnol and T.R. Stanley and K.D.E. Stokesbury and A. Torres and O. Tully and T. Vehanen and C. Watts and Qingyuan Zhao and W.J. Sutherland",

year = "2020",

month = dec,

day = "11",

doi = "10.1038/s41467-020-20142-y",

language = "English",

journal = "Nature Communications",

issn = "2041-1723",

publisher = "Springer Nature",

}

Christie, AP, Abecasis, D, Adjeroud, M, Alonso, JC, Amano, T, Anton, A, Baldigo, BP, Barrientos, R, Bicknell, JE, Buhl, DA, Cebrian, J, Ceia, RS, Cibils-Martina, L, Clarke, S, Claudet, J, Craig, MD, Davoult, D, Backer, AD, K. Donovan, M, Eddy, TD, França, FM, P.A. Gardner, J, P. Harris, B, Huusko, A, L. Jones, I, Kelaher, BP, Kotiaho, JS, López-Baucells, A, Major, HL, Mäki-Petäys, A, Martín, B, A. Martín, C, A. Martin, P, Mateos-Molina, D, McConnaughey, RA, Meroni, M, Meyer, CFJ, Mills, K, Montefalcone, M, Noreika, N, Palacín, C, Pande, A, Pitcher, CR, Ponce, C, Rinella, M, Rocha, R, Ruiz-Delgado, MC, Schmitter-Soto, JJ, Shaffer, JA, Sharma, S, Sher, AA, Stagnol, D, Stanley, TR, Stokesbury, KDE, Torres, A, Tully, O, Vehanen, T, Watts, C, Zhao, Q & Sutherland, WJ 2020, 'Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences', Nature Communications. https://doi.org/10.1038/s41467-020-20142-y

TY - JOUR

T1 - Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

AU - Christie, A.P.

AU - Abecasis, D.

AU - Adjeroud, M.

AU - Alonso, J.C.

AU - Amano, T.

AU - Anton, A.

AU - Baldigo, B.P.

AU - Barrientos, R.

AU - Bicknell, J.E.

AU - Buhl, D.A.

AU - Cebrian, J.

AU - Ceia, R.S.

AU - Cibils-Martina, L.

AU - Clarke, S.

AU - Claudet, J.

AU - Craig, M.D.

AU - Davoult, D.

AU - Backer, A. De

AU - K. Donovan, Mary

AU - Eddy, T.D.

AU - França, Filipe M.

AU - P.A. Gardner, Jonathan

AU - P. Harris, Bradley

AU - Huusko, A.

AU - L. Jones, Ian

AU - Kelaher, B.P.

AU - Kotiaho, J.S.

AU - López-Baucells, A.

AU - Major, H.L.

AU - Mäki-Petäys, A.

AU - Martín, B.

AU - A. Martín, Carlos

AU - A. Martin, Philip

AU - Mateos-Molina, D.

AU - McConnaughey, R.A.

AU - Meroni, M.

AU - Meyer, C.F.J.

AU - Mills, Kade

AU - Montefalcone, M.

AU - Noreika, N.

AU - Palacín, C.

AU - Pande, Anjali

AU - Pitcher, C.R.

AU - Ponce, C.

AU - Rinella, M.

AU - Rocha, R.

AU - Ruiz-Delgado, M.C.

AU - Schmitter-Soto, J.J.

AU - Shaffer, J.A.

AU - Sharma, Shailesh

AU - Sher, A.A.

AU - Stagnol, D.

AU - Stanley, T.R.

AU - Stokesbury, K.D.E.

AU - Torres, A.

AU - Tully, O.

AU - Vehanen, T.

AU - Watts, C.

AU - Zhao, Qingyuan

AU - Sutherland, W.J.

PY - 2020/12/11

Y1 - 2020/12/11

N2 - Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

AB - Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

UR - https://www.research.lancs.ac.uk/portal/en/publications/quantifying-and-addressing-the-prevalence-and-bias-of-study-designs-in-the-environmental-and-social-sciences(8fd03087-4ba9-433b-bb2d-91f543cd16e3).html

U2 - 10.1038/s41467-020-20142-y

DO - 10.1038/s41467-020-20142-y

M3 - Article (Academic Journal)

C2 - 33311448

SN - 2041-1723

JO - Nature Communications

JF - Nature Communications

ER -

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Abstract

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this