Projects | Evidence synthesis Lab

The Research Fingerprint project

Thu, 01 Jan 2026 00:00:00 +0000

Access the dashboard here.

The NEMO project

Fri, 01 Aug 2025 00:00:00 +0000

NEtwork and pairwise meta-analysis with Missing Outcome data

Project description

The problem with missing participant outcome data in evidence synthesis

Missing (participant) outcome data are ubiquitous in systematic reviews. In the absence of individual participant outcome data, it is challenging to address missing outcome data in their aggregate format. Therefore, (aggregate) missing outcome data have preoccupied many researchers, leading to a rather abundant published literature on that special topic mainly in the pairwise meta-analysis framework. While the methodology already developed in pairwise meta-analysis can be elaborated further to operate in a network of interventions, addressing missing outcome data in network meta-analysis holds an additional degree of complexity.

A call for research attention to this topic

The NEMO project was initiated in 2017 as a response to the scarce publication agenda on the methodology of missing outcome data in the network meta-analysis. For instance, there was no evidence on the prevalence, reporting and handling of missing outcome data in systematic reviews with network meta-analysis. Evidence on the performance of different methods to handle missing outcome data when conducting network meta-analysis was missing as well.

The first phase of the NEMO project provided empirical evidence on the reporting and handling of missing outcome data in networks of interventions, followed by a thorough empirical and simulation evaluation of the performance of different models to handle missing binary outcome data (for being the most prevalent outcome in evidence synthesis) in network meta-analysis.

The second phase of the NEMO project aimed to enrich and further advance the methodology for missing outcome data in systematic reviews. The research agenda included an extensive refinement of current modelling strategies for continuous missing outcome data that will offer, in addition, the possibility to learn about the missingness mechanisms in the collected trials. Furthermore, a user-appealing graphical approach was developed to visualise the sensitivity analysis results from making different plausible scenarios about the missingness mechanism based on the pattern-mixture model. A highlight of this work package was the development of a novel index to infer if the primary analysis results wedre robust to different scenarios about the informative missingness parameter. Lastly, all functions developed during the project were bundled into the rnmamod R package that is published in CRAN.

The NEMO project was funded by the German Research Foundation from 01/03/2017 to 31/12/2021.

Presentations

07/2021	42th Annual Conference of the International Society for Clinical Biostatistics (online), Lyon, France
	Spineli LM, Kalyvas C, Papadimitropoulou K. Quantifying the robustness of primary analysis results: a case study on missing outcome data in pairwise and network meta-analysis. Download presentation
09/2019	64. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS). Dortmund, 08.-11.09.2019
	Spineli LM, Kalyvas C. Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysis. doi: 10.3205/19gmds077; Download presentation
07/2019	40th Annual Conference of the International Society for Clinical Biostatistics, Leuven, Belgium
	Spineli LM, Kalyvas C. Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysis.
03/2019	Heidelberger Kolloquium, Institute of Medical Biometry and Informatics, University of Heidelberg, Germany
	Missing Outcome Data in Network Meta-analysis of Healthcare Interventions: a combination of a systematic overview, empirical and simulation study – The NEMO (NEtwork meta-analysis with Missing Outcome data) project. Download presentation
09/2018	63. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS). Osnabrück, 02.-06.09.2018
	Spineli LM. Comparison of methods to handle missing binary outcome data in network meta-analysis: an empirical study. doi: 10.3205/18gmds042; Download presentation
06/2018	Research Synthesis 2018 Conference in the University of Trier, Trier, Germany
	Spineli LM, Yepes-Nuñez J, Schünemann H. A systematic survey shows that reporting and handling of missing outcome data in networks of interventions is poor. Presentation; Download presentation
03/2018	64. Biometrisches Kolloquium 2018 (IBS-DR), Biometrie: gelebte Vielfalt 25.-28. März 2018 an der Goethe-Universität Frankfurt
	Spineli LM, Yepes-Nuñez J, Schünemann H. A systematic survey shows that reporting and handling of missing outcome data in networks of interventions is poor.

Publications

Original research

Spineli LM. Using network meta-analysis to predict the percentage of missing participants for a future trial. Res Methods Med Health Sci 2023; 4(4):140-149. Journal

Spineli LM. A revised framework to evaluate the consistency assumption globally in a network of interventions. Med Decis Making 2022; 42(5):637-648. PubMed

Spineli LM, Kalyvas C, Papadimitropoulou K. How robust are findings of pairwise and network meta-analysis in the presence of missing participant outcome data? BMC Med 2021; 19(1):323. PubMed

Spineli LM, Kalyvas C, Papadimitropoulou K. Quantifying the robustness of primary analysis results: A case study on missing outcome data in pairwise and network meta-analysis. Res Synth Methods 2021; 12(4):475-490. PubMed

Spineli LM, Papadimitropoulou K, Kalyvas C. Pattern-mixture model in network meta-analysis of binary missing outcome data: one-stage or two-stage approach? BMC Med Res Methodol 2021; 21(1):12. PubMed

Spineli LM, Kalyvas C, Papadimitropoulou K. Continuous(ly) missing outcome data in network meta-analysis: a one-stage pattern-mixture model approach. Stat Methods Med Res 2021; 30(4):958-975. PubMed

Spineli LM, Kalyvas C. Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysis. BMC Med Res Methodol 2020; 20(1):48. PubMed

Spineli LM, Kalyvas C, Pateras K. Participants’ outcomes gone missing within a network of interventions: Bayesian modeling strategies. Stat Med 2019; 38(20):3861-3879. PubMed

Spineli LM. An empirical comparison of Bayesian modelling strategies for missing binary outcome data in network meta-analysis. BMC Med Res Methodol 2019; 19(1):86. PubMed

Spineli LM, Yepes-Nuñez JJ, Schünemann HJ. A systematic survey shows that reporting and handling of missing outcome data in networks of interventions is poor. BMC Med Res Methodol 2018; 18(1):115. PubMed

Spineli LM. Modeling missing binary outcome data while preserving transitivity assumption yielded more credible network meta-analysis results. J Clin Epidemiol 2019; 105:19-26. PubMed

R package

Spineli LM. Rnmamod: Bayesian Network Meta-Analysis with Missing Participants. R package version 0.5.0. 2025. https://CRAN.R-project.org/package=rnmamod [creator and maintainer]. GitHub development version in https://github.com/LoukiaSpin/rnmamod.

The TRACE-NMA project

Fri, 01 Aug 2025 00:00:00 +0000

TRAnsitivity and Consistency Evaluation in Network Meta-Analysis

Project description

The problem with the exchangeability assumption in evidence synthesis

The explosive rate of such published reviews with multiple interventions attests to the popularity of network meta-analysis in the broad medical community. Nevertheless, the quality of the conclusions delivered to the end-users strongly depends on whether the exchangeability assumption, underlying network meta-analysis, is valid. The exchangeability assumption refers to the similarity of the synthesised evidence, based on epidemiological and statistical grounds, and is broadly known as transitivity and consistency. There is a vast literature on the concepts and methods that relate to the transitivity and consistency assumptions. However, the findings of several empirical studies on the reporting and evaluation quality of these assumptions have been underwhelming. Furthermore, most of these empirical studies have focused mainly on networks of three interventions. Therefore, their findings cannot be generalised to reviews that include complex networks of interventions. Another major limitation of these studies is their undue reliance on the p-value to infer the similarity of the synthesised evidence.

A call for research attention to this topic

Thorough improvement in the reporting and evaluation of the transitivity and consistency assumptions requires developing objective and validated methods to engage the authors of systematic reviews in that direction. The TRACE-NMA project was initiated in 2021 to promote the systematic evaluation of the transitivity and consistency assumptions while using proper methods to provide high-quality conclusions to the end-users. Initially, a systematic literature research was conducted to update the evidence on the reporting quality and evaluation of the transitivity assumption in complex networks with systematic reviews published between January 2011 and April 2015. Then, novel approaches were developed to judge the epidemiological and statistical similarity of the collated evidence objectively without undue reliance on statistical tests. Specifically, the agglomerative hierarchical clustering was proposed for the transitivity evaluation based on aggregate study-level characteristics that may act as important effect modifiers. The hierarchical clustering was grounded on the dissimilarities among all trial pairs in a connected network using Gower’s dissimilarity coefficient. In the context of consistency evaluation, the Kullback-Leibler divergence between the direct and indirect estimates of target comparisons was proposed to judge the local inconsistency as potentially low or material, particularly when statistical tests for local inconsistency are not conclusive.

The TRACE-NMA project was funded by the German Research Foundation from 01/01/2022 to 31/12/2024.

Presentations

03/2025	26. Jahrestagung des Netzwerks Evidenzbasierte Medizin. Freiburg, 26.-28.03.2025
	Spineli LM. Local inconsistency detection using the Kullback-Leibler divergence measure. doi: 10.3205/25ebm074; Download poster
09/2023	68. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS). Heilbronn, 17.-21.09.2023
	Spineli LM. Hierarchical clustering for the evaluation of transitivity assumption in a network of interventions. doi: 10.3205/23gmds078; Download presentation
08/2023	44th Annual Conference of the International Society for Clinical Biostatistics (hybrid), 27-31/08/2023, Milan, Italy
	Spineli LM. Hierarchical clustering for the evaluation of transitivity assumption in a network of interventions. Download presentation
03/2023	Evidence Synthesis and Meta-Analysis in R Conference (ESMARConf), online, 27-31/03/23
	Spineli LM, Kalyvas C, Papadimitropoulou K. Teaser Tutorial on Evidence Synthesis using the ‘rnmamod’ R Package. YouTube video
08/2022	67. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS), 13. Jahreskongress der Technologie- und Methodenplattform für die vernetzte medizinische Forschung e.V. (TMF). Kiel, 21.-25.08.2022
	Spineli LM, Kalyvas C, Seide S, Papadimitropoulou K. Evaluating the quality of reporting the transitivity assumption in complex networks of interventions. doi: 10.3205/22gmds077; Download presentation
08/2022	43th Annual Conference of the International Society for Clinical Biostatistics (hybrid), Newcastle, UK
	Spineli LM, Kalyvas C, Seide S, Papadimitropoulou K. Evaluating the quality of reporting the transitivity assumption in complex networks of interventions. Download poster

Publications

Original research

Spineli LM, García-Sierra AM, Yepes-Nuñez JJ. A database to initiate methodological advances in the evaluation of transitivity assumption in network meta-analysis: qualitative features and limitations of the tracenma R package. BMC Med Res Methodol 2025; 25(1):183. PubMed

Spineli LM, Papadimitropoulou K, Kalyvas C. Exploring the Transitivity Assumption in Network Meta-Analysis: A Novel Approach and Its Implications. Stat Med 2025; 44(7):e70068. PubMed

Spineli LM. An empirical study on 209 networks of treatments revealed intransitivity to be common and multiple statistical tests suboptimal to assess transitivity. BMC Med Res Methodol 2024; 24(1):301. PubMed

Spineli LM. Local inconsistency detection using the Kullback-Leibler divergence measure. Syst Rev 2024; 13(1):261. PubMed

Spineli LM, Kalyvas C, Yepes-Nuñez JJ, García-Sierra AM, Rivera-Pinzón DC, Seide SE, Papadimitropoulou K. Low awareness of the transitivity assumption in complex networks of interventions: a systematic survey from 721 network meta-analyses. BMC Med 2024; 22(1):112. PubMed

R package

Spineli LM. Tracenma: Database for Developing Transitivity Methodology in Network Meta-Analysis. R package version 0.1.1. 2025. https://CRAN.R-project.org/package=tracenma [creator and maintainer]. GitHub development version in https://github.com/LoukiaSpin/tracenma.