Design characteristics, risk of bias, and reporting of randomised controlled trials supporting European Medicines Agency approvals of cancer drugs, 2014-2016: Cross-sectional analysis

Objective: To examine the design characteristics, risk of bias, and reporting adequacy of pivotal randomised controlled trials (RCTs) of cancer drugs approved by the European Medicines Agency (EMA).

Design: Cross-sectional analysis.

Setting: European regulatory documents, clinical trial registry records, protocols, journal publications, and supplementary appendices.

Eligibility criteria: Pivotal RCTs of new cancer drugs approved by the EMA between 2014 and 2016.

Main outcome measures: Study design characteristics (randomisation, comparators, endpoints), risk of bias using the revised Cochrane tool (bias arising from the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result), and reporting adequacy (completeness and consistency of information in trial protocols, publications, supplementary appendices, clinical trials registries, and regulatory documents), collected by three researchers independently.

Results: Between 2014 and 2016, 32 new cancer drugs were approved by the EMA on the basis of 54 pivotal studies. Of these, 41 (76%) were RCTs and 13 (24%) were either non-randomised or single-arm studies. 39/41 RCTs had available publications and were included in our study. Only 10 RCTs (26%) measured overall survival as either a primary or co-primary endpoint, with the remaining trials evaluating surrogate measures such as progression-free survival and response rates. Overall, we judged 19 RCTs (49%) to be at high risk of bias for their primary outcome. Concerns due to missing outcome data (n=10) and measurement of the outcome (n=7) were the most common domains leading to high risk of bias judgments. Fewer RCTs evaluating overall survival as their primary endpoints were at high risk of bias compared to those evaluating surrogate efficacy or safety endpoints (2/10 [20%] versus 16/29 [55%], respectively). When we considered information available in regulatory documents and the scientific literature separately, our overall risk of bias judgments differed for 8 RCTs (21%), reflecting reporting inadequacies in both sources of information. Regulators identified additional deficits beyond the domains captured in risk of bias assessments for 10 drugs (31%). These included magnitude of clinical benefit, inappropriate comparators, and non-preferred study endpoints, which were not disclosed as limitations in scientific publications.

Conclusions: Most pivotal studies forming the basis of EMA approval of new cancer drugs between 2014 and 2016 were RCTs. However, almost half of these were judged to be at high risk of bias based on their design, conduct or analysis, some of which might be unavoidable due to the complexity of cancer trials. Both regulatory documents and the scientific literature had gaps in their reporting. Journal publications did not acknowledge the key limitations of the available evidence identified in regulatory documents.