Clinical Prediction Model Development and Validation for the Detection of Newborn Sepsis, Diagnostic Research Protocol

Introduction

Neonatal sepsis is an infection-related state of systemic inflammation. Infants that are 28 days or younger and have infection-related symptoms are categorized as having a clinical syndrome. This is demonstrated by systemic signs of infection and the isolation of a bacterial or other pathogen from the bloodstream, which results in an estimated 750,000 deaths annually.¹

Neonatal sepsis is a clinical condition that can cause multisystem organ failure, circulatory shock, and systemic infection symptoms. It is diagnosed in neonates younger than 28 days old. There are two types of newborn sepsis: early-onset neonatal sepsis (EONS) and late-onset neonatal sepsis (LONS). A typical definition of early-onset neonatal sepsis is an infection and sepsis that occurs within the first week to 24 hours of life.^2,3 LONS can develop up to 28 days, one month, or after the first week of life.^4,5 The definitions of EONS and LONS in the literature vary, but the majority classify EONS as occurring within the first 72 hours of life and LONS as occurring after this time period up to 28 days.^6,7

Simple clinical criteria were created by the World Health Organization (WHO) young baby research group to identify neonates who needed to be referred to a medical facility for the treatment of a severe bacterial illness.⁸ The Integrated Management of Neonatal and Childhood Illness (IMNCI) clinical algorithm has modified these criteria and incorporates the following clinical aspects to diagnose clinical neonatal sepsis: if the neonate had a temperature greater than 37.5°C or felt warm to the touch, convulsions (based on history), rapid breathing (more than 60 breaths per minute), severe chest in drawing, nasal flaring, grunting, bulging fontanels, pus draining from the ear, umbilical redness that reached the skin, felt cold (based on history), numerous or severe skin pustules, difficulty waking up, inability to be calmed within an hour.⁹

Despite recent improvements in healthcare facilities, sepsis is still one of the leading causes of morbidity and mortality in newborns globally. With mortality ranging from 11% to 19%, the estimated global burden of neonatal sepsis was 2202 (95% CI: 1099–4360) per 100,000 live births. 4,5 Neonatal mortality accounts for more than 40% of deaths in children under the age of five, killing 3.1 million newborns annually.^10,11 Despite the fact that infant health has significantly improved globally over the past 20 years, 2.5 million infants died in 2018, resulting in a global neonatal mortality rate of 18 deaths per 1000 live births.¹² In low- and middle-income countries (LMICs), prematurity (35%) and problems related to intrapartum (24%) as well as newborn sepsis account for the majority of these fatalities.¹³ Worldwide, 2202 cases of neonatal sepsis are anticipated to occur for every 100,000 live births, with a case fatality rate of 11 to 19%. Furthermore, it significantly increases morbidity among survivors because conditions like cerebral palsy, impaired vision or hearing, and aberrant neurodevelopment can last well past the newborn stage.¹⁴

Despite significant reductions in infant death by the Ethiopian government, neonatal mortality remains stable.¹⁵ The newborn death rate in Ethiopia is 30 per 1000 live births, according to the Ethiopian Demographic and Health Survey (EDHS) (2019). Neonatal sepsis has been found to be highly prevalent in Ethiopia. Ethiopia has a 45% overall newborn sepsis prevalence rate.^16,17 There are several established risk factors for newborn sepsis, including premature birth, low birth weight, maternal illnesses, early membrane rupture, extended work hours, and the setting of the neonatal intensive care unit (NICU).^18–20

Ethiopia is trying to achieve its Sustainable Development Goals by lowering newborn mortality to less than 12/1000 (SDG). Two examples are the development of medical facilities across the nation and the training of medical personnel. Since 2005, maternal and neonatal care services have improved significantly, prenatal care (28% to 74%), skilled midwives (6% to 50%), and institutional deliveries (5% to 48%).²¹ As part of the strategy, the development of clinical predictive models for diagnosing early neonatal sepsis is paramount.

The cryptic nature of newborn sepsis’ signs and symptoms makes diagnosis and prognosis challenging. The need for good biomarkers is fundamental to careful and timely diagnosis.²² Inadequate immunity in newborns makes them soldiers in overcoming death. Therefore, we need to consider new approaches to deal with the situation.

Nonspecific clinical symptoms that frequently overlap with non-communicable diseases, such as temperature instability, lethargy, poor eating, and dyspnea, make it difficult to diagnose newborn sepsis. The current gold standard for determining the pathogen from a typically sterile source (such as blood or CSF) is to diagnose newborn sepsis).²³ However, clinical sepsis (infants displaying sepsis-like symptoms despite having negative blood cultures) is a recognized condition, especially when the infant or mother has previously been exposed to blood cultures. Clinical sepsis can occur more frequently than sepsis that has been confirmed by blood cultures.²⁴

Consider whether a major illness is untreated or whether antibiotics are required before treating a suspected case of newborn sepsis. Antimicrobial resistance is a problem that is becoming increasingly widespread; according to one estimate, it is to blame for 31% of sepsis-related infant deaths globally. Additionally, a number of studies have demonstrated a link between prolonged antibiotic exposure and unfavorable newborn outcomes, including mortality and necrotizing enterocolitis.^25,26 To ensure proper antibiotic usage, minimize antimicrobial resistance, and increase neonatal survival, it is essential to correctly identify newborns that have neonatal sepsis.

Recent years have seen a lot of research interest in clinical predictive models, which are instruments that combine various characteristics (or predictors) to calculate the likelihood of a diagnostic or prognostic outcome.²⁷ There are several predictive models that estimate the risk of neonatal sepsis based on various clinical features, risk factors, and/or laboratory tests.²⁸

Therefore, models that do not consider test findings as predictors imply that basic laboratory tests are frequently unavailable and that primary care and clinical management of neonates are frequently restricted to distant, lower-level healthcare professionals with little administrative support. This is crucial in low- and middle-income countries because they may entrust, therefore, it is essential for early identification of newborn sepsis to design and validate clinical prediction models based on easily available data.

Significance of the Study

Although studies exist to identify risk factors for newborn sepsis, it is currently not possible to predict individual patient risk in clinical practice. In our setting, we expect that such a model could be used to predict the risk of newborn sepsis. In addition, it helps caregivers to rapidly select treatment for individual patients, making it more cost-effective by identifying high-risk patients who are likely to benefit from a particular intervention. Therefore, the results of the study may be used by clinicians.

Methods and Materials

Study Design

There will be an institution-based cross-sectional study design.

Study Setting and Study Period

Two of the twelve administrative zones in Amhara National Regional State (ANRS) are North Wollo and Wagehemira, which are both in Northeastern Ethiopia and have Sekota and Woldia as their respective capital cities. There are woredas and towns administrated under the wings of the two zones.

The Population of the Study

Source Population

All newborns admitted to the neonatal intensive care unit at Wagehemira and North Wollo Referral hospitals will be the study’s subjects.

Study Population

All neonates born between January 2020 and January 2022 who were admitted to and treated at the Comprehensive hospital of North Wollo and Wagehemira zone.

Eligibility Criteria

Includes all charts and ages for newborns up to 28 days admitted to Wagehemira NICU and North Wollo Referral Hospitals.

Variables

Outcome Variable

Neonatal sepsis (Yes/No).

Predictor Variables

Demographic Characteristics

Age in months, sex of newborn, residence of mother, and delivery place.

Maternal Factors

Parity, delivery mode, maternal illness, age of mother, history of UTI, ANC follow-up, history of maternal fever, history of drinking odorous alcohol, history of chorioamniotic, history of PROM, length of membrane rupture, length of labor, and past maternal medication use are all factors to be considered.

Newborn Related Variables

Weight at birth, gestational age, asphyxia at birth, and Apgar score.

Operational Definition

The main outcome of interest was neonatal sepsis among admitted Ethiopian neonates as diagnosed by the diagnostic criteria of neonatal sepsis by the established Integrated Management of Neonatal and Childhood Illness (IMNCI) guideline. According to the guideline, a neonate was recorded as septic when it had two or more of the following clinical features along with ≥2 of the subsequent hematological criteria: persistent fever (≥37.5 °C) or persistent hypothermia (≤35.5 °C) for more than 1 h, fast breathing (≥60 breath per minute), severe chest in drawing, grunting, not feeding well, movement only when stimulated, bulged fontanel, convulsion, lethargic or unconsciousness along with ≥2 of the hematological criteria such as total leukocyte count (<4000 or >12,000 cells/mm3), absolute neutrophil count (<1500 cells/mm3 or >7500 cells/mm3), platelet count (<150 or >450 cells/mm3), and random blood sugar (<40 mg/dl or >125 mg/dl).²⁹

Early onset of sepsis: A sepsis develops between birth and seven days old.

Late onset of sepsis: A sepsis develops between the ages of 8 and 28 days.³⁰

Sample Size Determination

Because there were no prior estimations, it was impossible to calculate the sample size for the development study. As a result, the sample size for logistic regression prediction models was estimated using the general rule of thumb of 10 events per variable.³¹ The expected number of events for the study will be 210 because there were 21 candidate variables taken into consideration, and each candidate variable had 10 events. The required sample size will be determined as follows: n= 210*100/45=467 considering that a study on the prevalence of newborn sepsis in Ethiopia indicated that it was 45%.

Sampling Method and Procedures

Neonates will be chosen using a simple random sample procedure, using their NICU medical registration number. Firstly, based on patient flow, we will make a proportional allocation of the sample size. Following that, the study will incorporate the newborn records of those who match the inclusion criteria. Finally, a randomly generated number created by a computer will be used to choose the study unit.

Data Collection Tools and Procedures

Data is collected through chart review using a structured checklist. Checklists will be developed after reviewing of various relevant literatures.^{11,16,20,21,32} The data includes socio-demographic data: age of the newborn, sex of the newborn, place of residence, and place of birth. Parity, method of delivery, maternal disease, maternal age, history of UTI, ANC follow-up, history of maternal fever, history of smelling alcohol, history of chorioamnionitis, history of meconium-stained amniotic fluid, history of PROM, duration of membrane rupture Duration of labour, duration of labor and previous maternal medication use are examples of maternal factors related to neonates: birth weight, gestational age, asphyxia at birth and Apgar score recorded using a structured checklist by chart review. In this study, neonatal sepsis is defined as when a medical diagnosis of the neonate is written down in the infant’s medical record chart as “neonatal sepsis.” The data collection method will involve a total of eight health professionals. Six health professionals with a nursing diploma work as data collectors, while two professionals with a Bachelor of Science (BSc) in nursing degree work as supervisors.

Data Management and Analysis

For statistical analysis, the data are entered into a software program (EPI DATA, version 3.2) and exported to the statistical programming language R, version 4.0.4. We assumed the data are missing at random, and the missing results will be imputed for all variables evaluated in the prediction model.³³ The validity of the assumption of missing at random (MAR) will be evaluated using sensitivity analysis. Bivariable logistic regression will be used to create the model by providing information on the relationships between each potential predictor and preterm birth. In the multivariate logistic regression model, significant factors (P<0.05) are retained while including variables with (P<0.25) from the bivariate analysis.

To present the results of significant predictors, coefficients, odds ratios (ORs), and 95% confidence intervals (CIs) will be used (CI).

We will compute the area under the ROC curve (discrimination) and the calibration plot to assess the model’s correctness (calibration). The AUC is between 0.5 (no predictive power) and 1 (perfect discrimination).³⁴ The bootstrapping method will be used to correct the regression coefficients, ORs with a 95% confidence level, and the AUC for overfitting or overconfidence.

To make internal validation, we will compute 1000 random bootstrap³⁵ samples with replacement of all predictors in the data. The predictive performance of the model after bootstrapping is taken into account as the performance that may be anticipated when the model is applied to upcoming populations with comparable characteristics. Neonatal sepsis prediction scores are created by rounding each model coefficient to the nearest whole integer and dividing by its lowest value. The total points will then be rounded to the closest integer. After then, each person’s overall score will be calculated by allocating points for each variable present and adding them together.

Risk Score Development

We can easily create a score that can be used to forecast neonatal sepsis by dividing each model coefficient by the lowest coefficient.

The total points are then rounded to the closest integer. The final score for each person will be calculated by allocating points for each variable present and putting them together. A high or low risk of newborn sepsis might be assigned to each neonate thanks to the score’s transformation into a binary score.

To evaluate the scoring system’s discriminatory power, the receiver operating characteristic curve (ROC) and area under the curve (AUC) will be shown.

Data Quality Assurance

Data collectors and supervisors will receive a day’s worth of training on the purpose of the study, the questionnaire’s content, and how to complete it in order to maintain the quality of the data. Afterward, a pretest will be done on 5% (24) of the medical records of neonates at Lalibela General Hospital which is found in Northern Ethiopia. After the pretest, some adjustments will be done accordingly. The questionnaire will be developed in English.