Adjusting trial results for biases in meta-analysis: combining data-based evidence on bias with detailed trial assessment

Flaws in the conduct of randomised trials can lead to biased estimation of the intervention effect. Methods for adjustment of within-trial biases in meta-analysis include use of empirical evidence from an external collection of meta-analyses, and use of expert opinion informed by assessment of detailed trial information. Our aim is to present methods to combine these two approaches in order to gain the advantages of both. We make use of the risk of bias information routinely available in Cochrane reviews, by obtaining empirical distributions for the bias associated with particular bias profiles (combinations of risk of bias judgements). We propose three methods: (i) formal combination of empirical evidence and opinion in a Bayesian analysis; (ii) asking experts to give an opinion on bias informed by both summary trial information and a bias distribution from the empirical evidence, either numerically or by (iii) selecting areas of the empirical distribution. The methods are demonstrated through application to two example binary outcome meta-analyses. Bias distributions based on opinion informed by trial information alone were most dispersed on average, and those based on opinions obtained by selecting areas of the empirical distribution were narrowest. Although the three different methods for combining empirical evidence with opinion vary in ease and speed of implementation, they yielded similar results in the two examples.