Model-based meta-analysis in drug and device development and the added value of data sharing

Meta-analysis – what for?

Swiss stakeholders and international funders generally agree that new clinical trials should be justified by a systematic review of the evidence that includes meta-analysis assessment. This highlights the importance of systematic reviews and meta-analysis in the future.¹ This standpoint is in line with the changes in the regulatory landscape in Europe and beyond, which require methodologically sound systematic reviews and meta-analysis for drug and medical devices approval (the results of which should be included in a clinical evaluation report) that should be updated on a regular basis (e.g. every two to three years). Model-based meta-analysis is an emerging methodology that quantifies the evidence on efficacy, tolerability, and safety in an unbiased manner in order to support better decision-making in clinical development and drug and device development. Meta-analysis is a systematic review of evidence that includes a mathematical combination of results from different studies. Combining and pooling results from different animal and human studies, which are selected based on a comprehensive search of the literature with well-defined inclusion and exclusion criteria, can lead to an unbiased and more comprehensive evaluation of the evidence. It also increases precision due to a larger sample size. Furthermore, the synthesis of study results across different studies can resolve research questions left unanswered by individual studies and explore factors that can explain conflicting results.

What information is essential for performing meta-analysis?

To perform a meta-analysis, it is important to start with a team that has different types of expertise, including clinical and methodological knowledge of study design and meta-analysis. Next, a focused research question with a defined exposure and/or intervention, study population, and outcome should be formulated. Well-defined inclusion and exclusion criteria are important in order to help select the final studies that will contribute to the analysis. This, together with a systematic search of different bibliographic databases, assures a thorough and unbiased investigation of the literature in the research topic of interest (see the selection procedure in Figure 1).² Once studies have been selected that fulfil the eligibility criteria and should be included in the final analysis, data from individual studies need to be extracted in order to pool and analyse the results (see example in Table 1). The information extracted should be based on an a priori decision and will depend on the research questions and the subgroup analysis that will be conducted as well as potential factors being explored as sources of heterogeneity (differences in results across studies). In general, it is important to extract information related to the author’s name, publication year, study title, study design, location, study name, duration (follow-up time), number of participants, percentage of female participants, number of events, age mean and standard deviation of participants, obesity, ethnicity, definition and/or assessment of the exposure and outcome, levels of adjustments, analysis type, estimates and their 95% confidence intervals or standard error for each adjustment level, funding (private vs. public), and the risk of bias and/or quality assessment. The extraction must be brief with clear abbreviations, consistent definitions, and the same units. It is advisable to have at least two independent researchers extract the data in order to minimise errors and potential biases. In addition, training and orientation should be provided prior to the extraction. To pool the results, information on summary statistics from each individual study needs to be extracted, including the measure of association estimate (e.g. mean difference, odds ratio, relative risk, or hazard ratio), the 95% confidence interval, and/or the standard error.

Conditions for using data

Because the meta-analysis of literature uses data from published articles, there are no specific conditions for using the data. Extraction can be done with electronic forms or a database management software such as Microsoft Access or REDCap. Using one of the many data systems available (e.g. EPPI-Reviewer, Systematic Review Data Repository (SRDR), DistillerSR (Evidence Partners), or Doctor Evidence) can be a more sophisticated alternative since they can be integrated with the title and abstract, allow for full-text screening, and export data directly into analysis software. However, it requires an investment to set up these commercial systems and train data extractors. It is recommended to share the data collected as part of systematic reviews and meta-analyses. Sharing extracted data has several potential benefits: it can minimise redundant work, improve the quality and efficiency of future reviews and meta-analyses, and support additional analyses.³

Data sharing: Individual patient data meta-analysis

Sharing research data at the individual patient level is a tremendous opportunity to perform more impactful meta-analysis with greater generalisability and increase the quality of evidence. Performing meta-analysis on individual patient data (IPD) relies on access to shared raw data from studies included in the meta-analysis. This allows for the reanalysis of data sets and the combination of the information from different studies, which are typically clinical trials. When accessible, such an approach has many advantages, which makes IPD meta-analysis the gold standard for systematic reviews.⁴

Generally, the first step to accessing IPD is convincing the sponsor-investigators from all of the different studies (including those studies that may have not been published) to share IPD on a scientific collaboration basis. Their active participation is also a way to get much more knowledge from the leading investigator on board. Additionally, access to individual patient data may override the limitation of poorly reported studies that otherwise would not be included in an aggregated data meta-analysis, thus reintroducing the data of numerous patients that would have been excluded. Using IPD also allows researchers to work on data quality, impute missing data, redefine and homogenise exposure measures and the criteria of judgment (duration of follow-up, composite outcomes, etc.), better assess the effect size of an intervention, gain a lot of analysis power that allows multivariate and time-to-event analysis, and introduce sensitivity analysis within their meta-analysis. Of course, all data transfer and sharing should be covered by data sharing agreements and should fully respect data confidentiality and protection standards.

A two-step approach is normally used for IPD meta-analysis: first, each study data set is reanalysed according to a homogenised statistical analysis plan; second, a standard random- or fixed-effects meta-analysis is performed based on an a priori decision. When performing time-to-event analysis or subgroups and interactions analysis, a one-step multi-level modelling approach is preferred that takes into account both study-level and patient-level covariates.

If IPD is not accessible for all studies, some analysis techniques may allow a combination of IPD and aggregated data in the meta-analysis. Thus, the advantages of IPD meta-analysis clearly outweigh meta-analysis with only aggregated data. This clearly demonstrates the enormous benefits of sharing research data at the individual participant level – as long as one ensures that processes are in place that uphold the most stringent confidentiality and data protection standards.⁵

Relevance of meta-analysis

Meta-analysis provides the highest levels of evidence and is used to generate guidelines, policies, and evidence-based practices in health care. Meta-analysis can help quantify the average effect of certain interventions (e.g. a drug or device), identify potential side effects, and compare the efficacy of different interventions. In general, medical and public health questions are studied more than once and are thus addressed by different research groups and conducted in different populations and locations. By combining data from different studies and heterogeneous populations, meta-analysis can provide insights into the generalisability of the findings, and it can identify subgroups of a population that can benefit the most from an intervention. By identifying these knowledge gaps, primary research such as clinical trials may be designed and conducted to fill such gaps. Thus, meta-analysis can inform and improve the design of future clinical trials and help guide drug and device development decisions.

References

1. McLennan S et al. (2021) Barriers and facilitating factors for conducting systematic evidence assessments in academic clinical trials. JAMA Network Open 4(11):e2136577. doi: 10.1001/jamanetworkopen.2021.36577
2. Muka T et al. (2020) A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in clinical research. European Journal of Epidemiology 35(49–60). doi: 10.1007/s10654-019-00576-5
3. Wolfenden L et al. (2016) Time to consider sharing data extracted from trials included in systematic reviews. Systematic Reviews 5:185. doi: 10.1186/s13643-016-0361-y
4. Tierney JF, Stewart LA, and Clarke M (2022) Chapter 26: Individual participant data. In Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, and Welch VA (eds) Cochrane Handbook for Systematic Reviews of Interventions (version 6.3, updated February 2022). Cochrane. Accessed 30 April 2022: https://training.cochrane.org/handbook/current/chapter-26
5. Stewart LA et al. (2015) Preferred reporting items for a systematic review and meta-analysis of individual participant data: The PRISMA-IPD Statement. JAMA 313(16):1657-1665. doi: 10.1001/jama.2015.3656