Impact of neonatal sepsis calculators on diagnostic accuracy and antimicrobial stewardship: A systematic review and meta-analysis

Visuddho Visuddho; Fan Maitri Aldian; Jason Oktavian Hartanto; Bendix Samarta Witarto; Mahendra Tri Arif Sampurna; Martono Tri Utomo; Risa Etika; Dina Angelika; Kartika Darma Handayani; Wurry Ayuningtyas; Abyan Irzaldy; Rizalya Dewi; Muhammad Robi; Setya Dewi Lusyati; Adhi Teguh Perma Iskandar

doi:10.23750/abm.v96i2.16929

Authors

Visuddho Visuddho Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia; Neonatal Research Group Surabaya, Faculty of Medicine, Universitas Airlangga https://orcid.org/0000-0002-6780-6207
Fan Maitri Aldian Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia https://orcid.org/0000-0001-7718-0247
Jason Oktavian Hartanto Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia https://orcid.org/0009-0004-8061-6952
Bendix Samarta Witarto Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia https://orcid.org/0000-0001-7837-756X
Mahendra Tri Arif Sampurna Department of Child Health, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia; Department of Child Health, Universitas Airlangga Hospital, Surabaya, Indonesia; Neonatal Research Group Surabaya, Faculty of Medicine, Universitas Airlangga; Research Center in Advancing Community Healthcare (REACH), Universitas Airlangga, Surabaya, Indonesia https://orcid.org/0000-0002-7529-8142
Martono Tri Utomo Department of Child Health, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia; Department of Child Health, Dr Soetomo General Academic Hospital, Surabaya, Indonesia; Neonatal Research Group Surabaya, Faculty of Medicine, Universitas Airlangga https://orcid.org/0000-0003-4828-2134
Risa Etika Department of Child Health, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia; Department of Child Health, Dr Soetomo General Academic Hospital, Surabaya, Indonesia; Neonatal Research Group Surabaya, Faculty of Medicine, Universitas Airlangga https://orcid.org/0000-0001-5670-7775
Dina Angelika Department of Child Health, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia; Department of Child Health, Dr Soetomo General Academic Hospital, Surabaya, Indonesia; Neonatal Research Group Surabaya, Faculty of Medicine, Universitas Airlangga https://orcid.org/0000-0003-2603-3532
Kartika Darma Handayani Department of Child Health, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia; Department of Child Health, Dr Soetomo General Academic Hospital, Surabaya, Indonesia; Neonatal Research Group Surabaya, Faculty of Medicine, Universitas Airlangga https://orcid.org/0000-0001-5420-3872
Wurry Ayuningtyas Department of Child Health, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia; Department of Child Health, Dr Soetomo General Academic Hospital, Surabaya, Indonesia; Neonatal Research Group Surabaya, Faculty of Medicine, Universitas Airlangga https://orcid.org/0000-0002-5744-3860
Abyan Irzaldy Department of Public Health, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands https://orcid.org/0000-0003-4294-9858
Rizalya Dewi Budhi Mulia Women and Children Hospital, Pekanbaru, Indonesia https://orcid.org/0009-0006-5950-1952
Muhammad Robi Department of Child Health, Faculty of Medicine, Universitas Riau, Arifin Achmad General Hospital, Pekanbaru, Indonesia https://orcid.org/0009-0002-1929-1242
Setya Dewi Lusyati National Center for Women and Children’s Health Harapan Kita, Jakarta, Indonesia https://orcid.org/0000-0002-6514-7084
Adhi Teguh Perma Iskandar Department of Child Health, Faculty of Medicine, Universitas Indonesia, Dr. Cipto Mangunkusumo General Hospital, Jakarta, Indonesia https://orcid.org/0000-0002-5776-9692

Keywords:

Neonatal sepsis calculator, Early-onset neonatal sepsis, Antibiotic stewardship, Diagnostic accuracy, Meta-analysis, Neonatal infection, Sepsis, Antimicrobial stewardship, Neonatal sepsis diagnosis, Early detection of neonatal sepsis, Meta-analysis in sepsis, Sepsis diagnostic tools

Abstract

Background and Aim: Early-onset neonatal sepsis (EOS) is a life-threatening infection occurring within the first 72 hours of life, requiring timely treatment to optimize outcomes. Although group B streptococcus screening and intrapartum antibiotic prophylaxis have reduced EOS incidence, concerns regarding antibiotic overuse persist. The neonatal sepsis calculator, developed by Kaiser Permanente Northern California, is a risk-based prediction tool designed to improve antibiotic stewardship. This systematic review and meta-analysis evaluate its diagnostic accuracy and impact on antibiotic use.

Methods: A systematic literature search was conducted across PubMed, Scopus, Web of Science, ScienceDirect, and ProQuest (up to July 1, 2024), following the PRISMA guidelines (PROSPERO ID 567269). Studies assessing the neonatal sepsis calculator against blood culture results were included. Data on antibiotic usage with and without calculator implementation were extracted. Meta-analysis of diagnostic accuracy was performed using STATA 16.0 to determine pooled sensitivity, specificity, diagnostic odds ratio (DOR), and area under the curve (AUC). Proportional and binary meta-analysis using RStudio 4.4.1 summarized antibiotic usage changes.

Results: From 1,523 records, 21 observational studies were included. The neonatal sepsis calculator showed a pooled sensitivity of 68% (95% CI 49–82%) and specificity of 78% (95% CI 57–90%), with an AUC of 0.79 (95% CI 0.75–0.82) and a DOR of 7.25 (95% CI 2.49–21.08). Its implementation significantly reduced antibiotic use (OR 0.045, 95% CI 0.009–0.236, p=0.001).

Conclusion: The neonatal sepsis calculator demonstrates good diagnostic accuracy and supports antibiotic stewardship. Future refinements incorporating neonatal factors may enhance its predictive performance.

References

Hornik CP, Fort P, Clark RH, et al. Early and late onset sepsis in very-low-birth-weight infants from a large group of neonatal intensive care units. Early Hum Dev. 2012;88:S69-S74. doi: 10.1016/S0378-3782(12)70019-1.

Simonsen KA, Anderson-Berry AL, Delair SF, et al. Early-Onset Neonatal Sepsis. Clin Microbiol Rev. 2014;27:21-47. doi: 10.1128/cmr.00031-13.

Edwards MS, Gonik B. Preventing the broad spectrum of perinatal morbidity and mortality through group B streptococcal vaccination. Vaccine. 2013;31:D66-D71. doi: 10.1016/j.vaccine.2012.11.046.

Fleischmann C, Reichert F, Cassini A, et al. Global incidence and mortality of neonatal sepsis: a systematic review and meta-analysis. Arch Dis Child. 2021;106:745. doi: 10.1136/archdischild-2020-320217.

Shane AL, Sánchez PJ, Stoll BJ. Neonatal sepsis. Lancet. 2017;390:1770-1780. doi: 10.1016/s0140-6736(17)31002-4.

Procianoy RS, Silveira RC. The challenges of neonatal sepsis management. J Pediatr. 2020;96:80-86. doi: 10.1016/j.jped.2019.10.004.

Li Q-y, Wang D-y, Li H-t, et al. Screening-based and Risk-based Strategy for the Prevention of Early-onset Group B Streptococcus/Non-group B Streptococcus Sepsis in the Neonate: A Systematic Review and Meta-analysis. Pediatr Infect Dis J. 2020;39.

Narasimhan SR, Govindaswami B, Jegatheesan P, et al. Mother-Infant Separation, High Illness Acuity Admission and Antibiotics Use in Inborn Infants >34 weeks Gestation: A Decade of Experience from a Regional NICU. Pediatrics. 2019;144:662-662. doi: 10.1542/peds.144.2MA7.662.

Kuppala VS, Meinzen-Derr J, Morrow AL, et al. Prolonged Initial Empirical Antibiotic Treatment is Associated with Adverse Outcomes in Premature Infants. J Pediatr. 2011;159:720-725. doi: 10.1016/j.jpeds.2011.05.033.

Piantino JH, Schreiber MD, Alexander K, et al. Culture Negative Sepsis and Systemic Inflammatory Response Syndrome in Neonates. NeoReviews. 2013;14:e294-e305. doi: 10.1542/neo.14-6-e294.

Akangire G, Simpson E, Weiner J, et al. Implementation of the Neonatal Sepsis Calculator in Early-Onset Sepsis and Maternal Chorioamnionitis. Adv Neonatal Care. 2020;20.

Carola D, Vasconcellos M, Sloane A, et al. Utility of Early-Onset Sepsis Risk Calculator for Neonates Born to Mothers with Chorioamnionitis. J Pediatr. 2018;195:48-52.e41. doi: 10.1016/j.jpeds.2017.11.045.

Strunk T, Buchiboyina A, Sharp M, et al. Implementation of the Neonatal Sepsis Calculator in an Australian Tertiary Perinatal Centre. Neonatology. 2018;113:379-382. doi: 10.1159/000487298.

Kopec G, Collin M, Das A. Application of Kaiser Sepsis Calculator in culture-positive infants with early onset sepsis. World J Pediatr. 2021;17:429-433. doi: 10.1007/s12519-021-00446-9.

Scott PA, Lai M, Inglis GDT, et al. Neonatal early-onset sepsis calculator safety in an Australian tertiary perinatal centre. J Paediatr Child Health. 2022;58:863-867. doi: 10.1111/jpc.15860.

van der Weijden BM, Achten NB, Bekhof J, et al. Neonatal early-onset sepsis calculator recommended significantly less empiric antibiotic treatment than national guidelines. Acta Paediatr. 2020;109:2549-2551. doi: 10.1111/apa.15391.

Schmatz M, Srinivasan L, Grundmeier RW, et al. Surviving Sepsis in a Referral Neonatal Intensive Care Unit: Association between Time to Antibiotic Administration and In-Hospital Outcomes. J Pediatr. 2020;217:59-65.e51. doi: 10.1016/j.jpeds.2019.08.023.

Achten NB, Klingenberg C, Benitz WE, et al. Association of Use of the Neonatal Early-Onset Sepsis Calculator With Reduction in Antibiotic Therapy and Safety: A Systematic Review and Meta-analysis. JAMA Pediatr. 2019;173:1032-1040. doi: 10.1001/jamapediatrics.2019.2825.

Rohsiswatmo R, Hikmahrachim HG, Nadobudskaya DU, et al. Sepsis calculator to support antibiotic stewardship in early-onset neonatal sepsis: a meta-analysis. Paediatr Indones. 2018;58:286-297. doi: 10.14238/pi58.6.2018.286-97.

Deshmukh M, Mehta S, Patole S. Sepsis calculator for neonatal early onset sepsis - a systematic review and meta-analysis. J Matern Fetal Neonatal Med. 2021;34:1832-1840. doi: 10.1080/14767058.2019.1649650.

McInnes MDF, Moher D, Thombs BD, et al. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA. 2018;319:388-396. doi: 10.1001/jama.2017.19163.

Deeks J, Bossuyt P, Leeflang M, et al. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2.0. Available from: https://training.cochrane.org/handbook-diagnostic-test-accuracy/current (2023).

Methley AM, Campbell S, Chew-Graham C, et al. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579. doi: 10.1186/s12913-014-0579-0.

Whiting PF, Rutjes AWS, Westwood ME, et al. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann Intern Med. 2011;155:529-536. doi: 10.7326/0003-4819-155-8-201110180-00009.

Mandrekar JN. Receiver Operating Characteristic Curve in Diagnostic Test Assessment. J Thorac Oncol. 2010;5:1315-1316. doi: 10.1097/JTO.0b013e3181ec173d.

Lin L, Chu H. Meta-analysis of Proportions Using Generalized Linear Mixed Models. Epidemiology. 2020;31:713-717. doi: 10.1097/ede.0000000000001232.

Warn DE, Thompson SG, Spiegelhalter DJ. Bayesian random effects meta-analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales. Stat Med. 2002;21:1601-1623. doi: 10.1002/sim.1189.

Alhindi MY, Almalki FG, Al Saif S, et al. Evaluating a Modified Use of the Kaiser Permanente Early-onset Sepsis Risk Calculator to Reduce Antibiotic Exposure: a Retrospective Study. BMJ Paediatr Open. 2024;8. doi: 10.1136/bmjpo-2024-002597.

Arora V, Strunk D, Furqan SH, et al. Optimizing antibiotic use for early onset sepsis: A tertiary NICU experience. J Neonatal Perinatal Med. 2019;12:301-312. doi: 10.3233/npm-180075.

Begnaud CM Jr, Lemoine J, Broussard L, et al. A Quality Improvement Project to Reduce Antibiotic Utilization and Ancillary Laboratory Tests in the Appraisal of Early-Onset Sepsis in the NICU. J Pediatr Nurs. 2021;60:215-222. doi: 10.1016/j.pedn.2021.06.012.

Dhudasia MB, Mukhopadhyay S, Puopolo KM. Implementation of the Sepsis Risk Calculator at an Academic Birth Hospital. Hosp Pediatr. 2018;8:243-250. doi: 10.1542/hpeds.2017-0180.

Fernandes M, Winckworth L, Lee L, et al. Screening for early-onset neonatal sepsis on the Kaiser Permanente sepsis risk calculator could reduce neonatal antibiotic usage by two-thirds. Pediatr Investig. 2022;6:171-178. doi: 10.1002/ped4.12344.

Goel N, Shrestha S, Smith R, et al. Screening for early onset neonatal sepsis: NICE guidance-based practice versus projected application of the Kaiser Permanente sepsis risk calculator in the UK population. Arch Dis Child Fetal Neonatal Ed. 2020;105:118-122. doi: 10.1136/archdischild-2018-316777.

Gong CL, Dasgupta-Tsinikas S, Zangwill KM, et al. Early onset sepsis calculator-based management of newborns exposed to maternal intrapartum fever: a cost benefit analysis. J Perinatol. 2019;39:571-580. doi: 10.1038/s41372-019-0316-y.

He Y, Chen J, Liu Z, et al. Efficacy and safety of applying a neonatal early-onset sepsis risk calculator in China. J Paediatr Child Health. 2020;56:237-243. doi: 10.1111/jpc.14572.

Joshi NS, Gupta A, Allan JM, et al. Management of Chorioamnionitis-Exposed Infants in the Newborn Nursery Using a Clinical Examination-Based Approach. Hosp Pediatr. 2019;9:227-233. doi: 10.1542/hpeds.2018-0201.

Kopsidas I, Molocha NM, Kourkouni E, et al. Potential benefit from the implementation of the Kaiser Permanente neonatal early-onset sepsis calculator on clinical management of neonates with presumed sepsis. Eur J Pediatr. 2022;181:1001-1008. doi: 10.1007/s00431-021-04282-x.

Kuzniewicz MW, Puopolo KM, Fischer A, et al. A Quantitative, Risk-Based Approach to the Management of Neonatal Early-Onset Sepsis. JAMA Pediatr. 2017;171:365-371. doi: 10.1001/jamapediatrics.2016.4678.

Money N, Newman J, Demissie S, et al. Anti-microbial stewardship: antibiotic use in well-appearing term neonates born to mothers with chorioamnionitis. J Perinatol. 2017;37:1304-1309. doi: 10.1038/jp.2017.137.

Piyasena C, Galu S, Yoshida R, et al. Comparison of diagnoses of early-onset sepsis associated with use of Sepsis Risk Calculator versus NICE CG149: a prospective, population-wide cohort study in London, UK, 2020-2021. BMJ Open. 2023;13:e072708. doi: 10.1136/bmjopen-2023-072708.

Quintero-Carreño LM, Quintero-Palacios MA, Palacios-Ariza MA, et al. Agreement between an Early-Onset Neonatal Sepsis Risk Calculator and the Colombian Clinical Practice Guideline in Three Tertiary-Care Centers in Bogotá, Colombia. Am J Perinatol. 2024;41:e1197-e1204. doi: 10.1055/a-2001-9012.

Shakib J, Buchi K, Smith E, et al. Management of newborns born to mothers with chorioamnionitis: is it time for a kinder, gentler approach? Acad Pediatr. 2015;15:340-344. doi: 10.1016/j.acap.2014.11.007.

Sharma V, Adkisson C, Gupta K. Managing Infants Exposed to Maternal Chorioamnionitis by the Use of Early-Onset Sepsis Calculator. Glob Pediatr Health. 2019;6:2333794X19833711. doi: 10.1177/2333794X19833711.

Sloane AJ, Carola DL, Lafferty MA, et al. Management of infants born to mothers with chorioamnionitis: A retrospective comparison of the three approaches recommended by the committee on fetus and newborn. J Neonatal Perinatal Med. 2021;14:383-390. doi: 10.3233/npm-200531.

Snoek L, van Kassel MN, Krommenhoek JF, et al. Neonatal early-onset infections: Comparing the sensitivity of the neonatal early-onset sepsis calculator to the Dutch and the updated NICE guidelines in an observational cohort of culture-positive cases. EClinicalMedicine. 2022;44:101270. doi: 10.1016/j.eclinm.2021.101270.

Stipelman CH, Smith ER, Diaz-Ochu M, et al. Early-Onset Sepsis Risk Calculator Integration Into an Electronic Health Record in the Nursery. Pediatrics. 2019;144. doi: 10.1542/peds.2018-3464.

Dong Y, Basmaci R, Titomanlio L, et al. Neonatal sepsis: within and beyond China. Chin Med J. 2020;133:2219-2228. doi: 10.1097/cm9.0000000000000935.

Sweet DG, Carnielli VP, Greisen G, et al. European Consensus Guidelines on the Management of Respiratory Distress Syndrome: 2022 Update. Neonatology. 2023;120:3-23. doi: 10.1159/000528914.

Liu L, Johnson HL, Cousens S, et al. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet. 2012;379:2151-2161. doi: 10.1016/s0140-6736(12)60560-1.

Chan GJ, Lee AC, Baqui AH, et al. Risk of early-onset neonatal infection with maternal infection or colonization: a global systematic review and meta-analysis. PLoS Med. 2013;10:e1001502. doi: 10.1371/journal.pmed.1001502.

Giannoni E, Dimopoulou V, Klingenberg C, et al. Analysis of Antibiotic Exposure and Early-Onset Neonatal Sepsis in Europe, North America, and Australia. JAMA Netw Open. 2022;5:e2243691. doi: 10.1001/jamanetworkopen.2022.43691.

Capin I, Hinds A, Vomero B, et al. Are Early-Onset Sepsis Evaluations and Empiric Antibiotics Mandatory for All Neonates Admitted with Respiratory Distress? Am J Perinatol. 2020;39:444-448. doi: 10.1055/s-0040-1717070.

Wu M, Deng Y, Wang X, et al. Development of risk prediction nomogram for neonatal sepsis in Group B Streptococcus-colonized mothers: a retrospective study. Sci Rep. 2024;14:5629. doi: 10.1038/s41598-024-55783-2.

Murthy S, Godinho MA, Guddattu V, et al. Risk factors of neonatal sepsis in India: A systematic review and meta-analysis. PLoS One. 2019;14:e0215683. doi: 10.1371/journal.pone.0215683.

Roy P, Kumar A, Kaur IR, et al. Gender differences in outcomes of low birth weight and preterm neonates: the male disadvantage. J Trop Pediatr. 2014;60:480-481. doi: 10.1093/tropej/fmu042.

Elsmén E, Hansen Pupp I, Hellström-Westas L. Preterm male infants need more initial respiratory and circulatory support than female infants. Acta Paediatr. 2004;93:529-533. doi: 10.1080/08035250410024998.

Ingemarsson I. Gender aspects of preterm birth. BJOG. 2003;110 Suppl 20:34-38. doi: 10.1016/s1470-0328(03)00022-3.

Garafova A, Kornanova E, Chovancova D, et al. Relationships between antenatal corticosteroids and catecholamine blood pressure support in neonates: considering of maternal stress-related diseases. Stress. 2020;23:694-699. doi: 10.1080/10253890.2020.1806227.

Kent AL, Wright IM, Abdel-Latif ME. Mortality and adverse neurologic outcomes are greater in preterm male infants. Pediatrics. 2012;129:124-131. doi: 10.1542/peds.2011-1578.

O'Driscoll DN, McGovern M, Greene CM, et al. Gender disparities in preterm neonatal outcomes. Acta Paediatr. 2018;107:1494-1499. doi: 10.1111/apa.14390.

Belachew A, Tewabe T. Neonatal sepsis and its association with birth weight and gestational age among admitted neonates in Ethiopia: systematic review and meta-analysis. BMC Pediatr. 2020;20:55. doi: 10.1186/s12887-020-1949-x.

Lee CC, Feng Y, Yeh YM, et al. Gut Dysbiosis, Bacterial Colonization and Translocation, and Neonatal Sepsis in Very-Low-Birth-Weight Preterm Infants. Front Microbiol. 2021;12:746111. doi: 10.3389/fmicb.2021.746111.

Silveira RC, Procianoy RS, Dill JC, et al. Periventricular leukomalacia in very low birth weight preterm neonates with high risk for neonatal sepsis. J Pediatr (Rio J). 2008;84:211-216. doi: 10.2223/jped.1777.

Khadse SN, Ugemuge S, Singh C. Impact of Antimicrobial Stewardship on Reducing Antimicrobial Resistance. Cureus. 2023;15:e49935. doi: 10.7759/cureus.49935.