A systematic review of recent publications was conducted to assess the extent to which contemporary micro-level research on smallholders facilitates data re-use and knowledge synthesis. Following PRISMA standards for systematic review, 1,182 articles were identified (published between 2018 and 2020), and 261 articles were selected for review in full. The themes investigated were: (i) data management, including data source, variables collected, granularity, and availability of the data; (ii) the statistical methods used, including analytical approach and reproducibility; and (iii) the interpretation of results, including the scope and objectives of the study, development issues addressed, scale of recommendations made relative to the scale of the sample, and the audience for recommendations. It was observed that household surveys were the most common data source and tended to be representative at the local (community) level. There was little harmonization of the variables collected between studies. Over three quarters of the studies (77%) drew on data which was not in the public domain, 14% published newly open data, and 9% drew on datasets which were already open. Other than descriptive statistics, linear and logistic regression methods were the most common analytical method used (64% of articles). In the vast majority of those articles, regression was used as an explanatory tool, as opposed to a predictive tool. More than half of the articles (59%) made claims or recommendations which extended beyond the coverage of their datasets. In combination these two common practices may lead to erroneous understanding: the tendency to rely upon simple regressions to explain context-specific and complex associations; and the tendency to generalize beyond the remit of the data collected. We make four key recommendations: (1) increased data sharing and variable harmonization would enable data to be re-used between studies; (2) providing detailed meta-data on sampling frames and study-context would enable more powerful meta-analyses; (3) methodological openness and predictive modeling could help test the transferability of approaches; (4) more precise language in study conclusions could help decision makers understand the relevance of findings for policy planning. Following these practices could leverage greater benefits from the substantial investment already made in data collection on smallholder farms.
Bibliographical noteFunding Information:
This work was supported by an Alan Turing Institute PhD Studentship funded under EPSRC grant EP/N510129/1, and a University of Bristol Enhancing Knowledge Partnerships grant funded by the Quality-Related Global Challenges Research Fund allocation. JH and MW acknowledge the support of the CGIAR Research Program on Livestock and its donors. CW acknowledges funding via a EPSRC RSE Fellowship (EP/N018591/1).
The authors would like to thank Kate Robson-Brown and the University of Bristol Jean Golding Institute for their support, and Levi J. Wolf and Maria Paula Escobar Tello for their comments on early drafts of this review.
Copyright © 2021 Gorman, Browne, Woods, Eisler, van Wijk, Dowsey and Hammond.
- agricultural research
- best practices
- quantitative research
- smallholder agriculture
- systematic review