Meta-analysis with zero-event studies: a comparative study with application to COVID-19 data

Jia-Jin Wei; En-Xuan Lin; Jian-Dong Shi; Ke Yang; Zong-Liang Hu; Xian-Tao Zeng; Tie-Jun Tong

doi:10.1186/s40779-021-00331-6

Your Location：

Home >

Browse articles >

Meta-analysis with zero-event studies: a comparative study with application to COVID-19 data

METHODOLOGY | Updated：2023-02-28

- Meta-analysis with zero-event studies: a comparative study with application to COVID-19 data
- Military Medical Research Vol. 9, Issue 1, Pages: 126-137(2022)
- Affiliations：
  
  1.Department of Mathematics, Hong Kong Baptist University, Hong Kong, China
  2.Shenzhen Research Institute of Big Data, Shenzhen, China
  3.College of Mathematics and Statistics, Shenzhen University, Shenzhen, China
  4.Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan University, Wuhan, China
- Author bio：
  
  * tongt@hkbu.edu.hk
- Funds：
- DOI：10.1186/s40779-021-00331-6
  CLC：
- Published：2022-02
- Accepted：
Scan QR Code
Wei et al.: Meta-analysis with zero-event studies: a comparative study with application to COVID-19 data. Mil Med Res, 2021, 8: 41.
DOI：

Wei et al.: Meta-analysis with zero-event studies: a comparative study with application to COVID-19 data. Mil Med Res, 2021, 8: 41. DOI： 10.1186/s40779-021-00331-6.

摘要

Abstract

Background:

Meta-analysis is a statistical method to synthesize evidence from a number of independent studies

including those from clinical studies with binary outcomes. In practice

when there are zero events in one or both groups

it may cause statistical problems in the subsequent analysis.

Methods:

In this paper

by considering the relative risk as the effect size

we conduct a comparative study that consists of four continuity correction methods and another state-of-the-art method without the continuity correction

namely the generalized linear mixed models (GLMMs). To further advance the literature

we also introduce a new method of the continuity correction for estimating the relative risk.

Results:

From the simulation studies

the new method performs well in terms of mean squared error when there are few studies. In contrast

the generalized linear mixed model performs the best when the number of studies is large. In addition

by reanalyzing recent coronavirus disease 2019 (COVID-19) data

it is evident that the double-zero-event studies impact the estimate of the mean effect size.

Conclusions:

We recommend the new method to handle the zero-event studies when there are few studies in a meta-analysis

or instead use the GLMM when the number of studies is large. The double-zero-event studies may be informative

and so we suggest not excluding them.

关键词

Keywords

references

Borenstein M , Hedges LV , Higgins JPT , Rothstein HR . Introduction to Meta-Analysis . Chichester, UK : John Wiley & Son ; 2011 .

Ma LL , Wang YY , Yang ZH , Huang D , Weng H , Zeng XT . Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Mil Med Res . 2020 ; 7 : 7 .

Davey J , Turner RM , Clarke MJ , Higgins JPT . Characteristics of meta-analyses and their component studies in the cochrane database of systematic reviews: a cross-sectional, descriptive analysis . BMC Med Res Methodol . 2011 ; 11 : 160 .

Jackson D , Law M , Stijnen T , Viechtbauer W , White IR . A comparison of seven random–effects models for meta-analyses that estimate the summary odds ratio . Stat Med . 2018 ; 37 ( 7 ): 1059 - 85 .

Ren Y , Lin L , Lian Q , Zou H , Chu H . Real-world performance of meta-analysis methods for double-zero-event studies with dichotomous outcomes using the cochrane database of systematic reviews . J Gen Intern Med . 2019 ; 34 ( 6 ): 960 - 8 .

Vandermeer B , Bialy L , Hooton N , Hartling L , Klassen TP , Johnston BC , Wiebe N . Meta-analyses of safety data: a comparison of exact versus asymptotic methods . Stat Methods Med Res . 2009 ; 18 ( 4 ): 421 - 32 .

Kuss O . Statistical methods for meta-analyses including information from studies without any events–add nothing to nothing and succeed nevertheless . Stat Med . 2015 ; 34 ( 7 ): 1097 - 116 .

Chu DK , Akl EA , Duda S , Solo K , Yaacoub S , Schünemann HJ , COVID-19 Systematic Urgent Review Group Effort (SURGE) study authors . Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS- CoV-2 and COVID-19: a systematic review and meta-analysis . Lancet . 2020 ; 395 ( 10242 ): 1973 - 87 .

Jin YH , Cai L , Cheng ZS , Cheng H , Deng T , Fan YP , et al . A rapid advice guideline for the diagnosis and treatment of 2019 novel coronavirus (2019-nCoV) infected pneumonia (standard version) . MilMed Res . 2020 ; 7 : 4 .

Jin YH , Zhan QY , Peng ZY , Ren XQ , Yin XT , Cai L , et al . Chemoprophylaxis, diagnosis, treatments, and discharge management of COVID-19: An evidence-based clinical practice guideline (updated version) . Mil Med Res . 2020 ; 7 : 41 .

Borenstein M , Hedges LV , Higgins JP , Rothstein HR . A basic introduction to fixed-effect and random-effects models for meta–analysis . Res Synth Methods . 2010 ; 1 ( 2 ): 97 - 111 .

Lin E , Tong T , Chen Y , Wang Y . Fixed-effects model: the most convincing model for meta-analysis with few studies . Preprint at https://arxiv.org/abs/2002.04211.2020 https://arxiv.org/abs/2002.04211.2020 .

Xu C , Li L , Lin L , Chu H , Thabane L , Zou K , Sun X . Exclusion of studies with no events in both arms in meta-analysis impacted the conclusions . J Clin Epidemiol . 2020 ; 123 : 91 - 9 .

Agresti A . Categorical Data Analysis , 2nd Edition . Hoboken : John Wiley & Son ; 2003 .

Haldane JB . The estimation and significance of the logarithm of a ratio of frequencies . Ann Hum Genet . 1956 ; 20 ( 4 ): 309 - 11 .

Schwarzer G . meta: An r package for meta-analysis . R News . 2007 ; 7 : 40 - 5 .

Weber F , Knapp G , Ickstadt K , Kundt G , Glass Ä . Zero–cell corrections in random–effects meta–analyses . Res Synth Methods . 2020 ; 11 ( 6 ): 913 - 9 .

Sweeting MJ , Sutton AJ , Lambert PC . What to add to nothing? use and avoidance of continuity corrections in meta–analysis of sparse data . Stat Med . 2004 ; 23 ( 9 ): 1351 - 75 .

Carter RE , Lin Y , Lipsitz SR , Newcombe RG , Hermayer KL . Relative risk estimated from the ratio of two median unbiased estimates . J Royal Stat Soc: Ser C Appl Stat . 2010 ; 59 ( 4 ): 657 - 71 .

Chu H , Nie L , Chen Y , Huang Y , Sun W . Bivariate random effects models for meta-analysis of comparative studies with binary outcomes: methods for the absolute risk difference and relative risk . Stat Methods Med Res . 2012 ; 21 ( 6 ): 621 - 33 .

Chen Y , Hong C , Ning Y , Su X . Meta–analysis of studies with bivariate binary outcomes: a marginal beta–binomial model approach . Stat Med . 2016 ; 35 ( 1 ): 21 - 40 .

DerSimonian R , Laird N . Meta-analysis in clinical trials . Control Clin Trials . 1986 ; 7 ( 3 ): 177 - 88 .

Laird NM , Mosteller F . Some statistical methods for combining experimental results . Int J Technol Assess Heal Care . 1990 ; 6 ( 1 ): 5 - 30 .

Balduzzi S , Rücker G , Schwarzer G . How to perform a meta-analysis with R: a practical tutorial . Evid Based Ment Health . 2019 ; 22 ( 4 ): 153 - 60 .

Pettigrew HM , Gart JJ , Thomas DG . The bias and higher cumulants of the logarithm of a binomial variate . Biometrika . 1986 ; 73 ( 2 ): 425 - 35 .

Hartung J , Knapp G . A refined method for the meta–analysis of controlled clinical trials with binary outcome . Stat Med . 2001 ; 20 ( 24 ): 3875 - 89 .

Fattorini L . Applying the Horvitz-Thompson criterion in complex designs: a computer-intensive perspective for estimating inclusion probabilities . Biometrika . 2006 ; 93 ( 2 ): 269 - 78 .

Seber GAF . Statistical Models for Proportions and Probabilities . Heidelberg : Springer ; 2013 .

Bakbergenuly I , Kulinskaya E . Meta-analysis of binary outcomes via generalized linear mixed models: a simulation study . BMC Med Res Methodol . 2018 ; 18 ( 1 ): 70 .

McCullagh P . Sampling bias and logistic models . J R Stat Soc Ser B Stat Methodol . 2008 ; 70 ( 4 ): 643 - 77 .

Zeger SL , Liang KY , Albert PS . Models for longitudinal data: a generalized estimating equation approach . Biometrics . 1988 ; 44 ( 4 ): 1049 - 60 .

Lin L , Chu H . Meta-analysis of proportions using generalized linear mixed models . Epidemiology . 2020 ; 31 ( 5 ): 713 - 7 .

Lin L , Chu H . altmeta: Alternative Meta-Analysis Methods . 2020 . https://CRAN.R-project.org/package=altmeta https://CRAN.R-project.org/package=altmeta .

Ju J , Lin L , Chu H , Cheng LL , Xu C . Laplace approximation, penalized quasi-likelihood, and adaptive gauss-hermite quadrature for generalized linear mixed models: towards meta-analysis of binary outcome with sparse data . BMC Med Res Methodol . 2020 ; 20 ( 1 ): 152 .

Gronsbell J , Hong C , Nie L , Lu Y , Tian L . Exact inference for the random–effect model for meta–analyses with rare events . Stat Med . 2020 ; 39 ( 3 ): 252 - 64 .

Sarmanov O . Generalized normal correlation and two-dimensional fréchet classes . Sov Math Dokl . 1966 ; 7 : 596 - 9 .

Chen Y , Luo S , Chu H , Su X , Nie L . An empirical Bayes method for multivariate meta-analysis with an application in clinical trials . Commun Stat Theory Methods . 2014 ; 43 ( 16 ): 3536 - 51 .

Luo S , Chen Y , Su X , Chu H . mmeta: an R package for multivariate meta-analysis . J Stat Softw . 2014 ; 56 ( 11 ): 11 .

Jia P , Lin L , Kwong JSW , Xu C . Many meta-analyses of rare events in the cochrane database of systematic reviews were underpowered . J Clin Epidemiol . 2021 ; 131 : 113 - 22 .

Li QK , Rice K . Improved inference for fixed–effects meta–analysis of 2×2 tables . Res Synth Methods . 2020 ; 11 ( 3 ): 387 - 96 .

Bender R , Friede T , Koch A , Kuss O , Schlattmann P , Schwarzer G , Skipka G . Methods for evidence synthesis in the case of very few studies . Res Synth Methods . 2018 ; 9 ( 3 ): 382 - 92 .

Rice K , Higgins JP , Lumley T . A re–evaluation of fixed effect(s) meta–analysis . J R Stat Soc Ser A Stat Methodol . 2018 ; 181 ( 1 ): 205 - 27 .

Yang K , Kwan HY , Yu Z , Tong T . Model selection between the fixed-effects model and the random-effects model in meta-analysis . Stat Interface . 2020 ; 13 ( 4 ): 501 - 10 .

Friedrich JO , Adhikari NK , Beyene J . Inclusion of zero total event trials in meta-analyses maintains analytic consistency and incorporates all available data . BMC Med Res Methodol . 2007 ; 7 : 5 .

Doi SA , Furuya-Kanamori L , Xu C , Lin L , Chivese T , Thalib L . Questionable utility of the relative risk in clinical research: a call for change to practice . J Clin Epidemiol . 2020 . https://doi.org/10.1016/j.jclinepi.2020.08.019 https://doi.org/10.1016/j.jclinepi.2020.08.019 .

Xiao M , Chen Y , Cole SR , MacLehose R , Richardson D , Chu H . Is OR “portable” in meta-analysis? Time to consider bivariate generalized linear mixed model . 2020 . Preprint at https://www.medrxiv.org/content/10.1101/2020.11.05.20226811v1 https://www.medrxiv.org/content/10.1101/2020.11.05.20226811v1 .

Doi SA , Furuya-Kanamori L , Xu C , Chivese T , Lin L , Musa OA , Hindy G , Thalib L , Harrell Jr FE . The OR is “portable” but not the RR: time to do away with the log link in binomial regression . J Clin Epidemiol . 2021 . https://doi.org/10.13140/RG.2.2.31631.10407 https://doi.org/10.13140/RG.2.2.31631.10407 .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Meta-analysis of the success rate of heartbeat recovery in patients with prehospital cardiac arrest in the past 40 years in China

A meta-analysis of the success rates of heartbeat restoration within the platinum 10min among outpatients suffering from sudden cardiac arrest in China

Dynamic Response Analysis of Tunnel Foundation by Vehicle Vibration in Metro

Research on Synthetical 3 Factors Checking Method for the Ultimate Load Carrying Capacity of Steel Arch Bridges

Research on the Measurement System of the Attack Angle between Wheel and Rail Based on Image Identification

Related Author

Xiang-Min Gu

Shi-Bin Yao

Zhong-Jie He

Yong-Gang Wang

Zhi-Hui Li

Guang-Wen Cao

Jian-Hua Yin

Yi-Chun Cao

Related Institution

Tianjin Binhai New Area Center for Disease Control and Prevention

Emergency Department, Affiliated Hospital of Chifeng College

The Fourth Medical Center of Chinese PLA General Hospital

Beijing Platinum Ten Minutes Time-effect Emergency Technology Research Academy

Beijing Chaoyang District Jiangtai Community Health Service Center

⁰