A quantitative approach to evidence triangulation: development of a framework to address rigour and relevance

Triangulation is an approach to strengthening causal inference by integrating evidence from multiple sources. Most studies using triangulation have qualitatively examined whether different studies agree upon the presence of a causal effect, rather than estimated the effect by quantitatively integrating results. Here, we develop a framework for quantitative triangulation. We first address how to relate study specific research questions to an overall target causal question (relevance), and then assess the directions and magnitudes of bias in each study (rigour), before combining the results using meta-analysis adjusted for the biases.

We illustrate our framework by triangulating evidence from randomized controlled trials (RCTs), Mendelian randomization (MR) and conventional multivariable regression (MVR) to estimate the effect of beta-carotene on coronary heart disease (CHD) and cardiovascular disease (CVD). Five RCTs and one MR study showed little evidence of a causal relationship between beta-carotene and CHD (relative risk (RR)=1.00 with 95% CI=0.98 to 1.01 and RR=1.02 with 95% CI=0.98 to 1.07, respectively). 13 MVR studies indicated that high intake of beta-carotene reduces CHD risk (RR=0.83 with 95% CI0.76 to 0.91). After applying our framework, the three study designs agreed that there is little evidence of an effect of beta-carotene intake on the risk of CHD (RR=1.01 with 95% CI=0.99 to 1.02). Findings were similar for CVD.

Our framework shows how to address rigour and relevance quantitatively when triangulating evidence from different study designs. We highlight the importance of explicitly defining the target and study-specific research questions.