Development of a Clinical Prediction Model for Infective Endocarditis Among Patients with Undiagnosed Fever: A Pilot Case–Control Study

Introduction

Infective endocarditis (IE) is a diagnostic challenge due to its non-characteristic clinical manifestations and physical and laboratory findings.¹ IE may be inadequately diagnosed as fever of unknown origin,^2,3 which is partially due to difficulty in applying all the advanced medical modalities required to make a diagnosis according to the modified Duke criteria (m-DC), such as transesophageal echocardiography (TEE), computed tomography (CT), and magnetic resonance imaging (MRI) for all patients with undiagnosed fever (UF).⁴ This may lead to improper practice of prescribing antibiotics before performing necessary examinations including blood cultures, especially at hospitals or clinics without those modalities. Naturally, administration of antibiotics before obtaining blood cultures could lead to failure to detect causative bacteria, causing a delayed diagnosis with poor prognosis of IE.^5,6 Therefore, predicting patients with a high probability of IE, using data routinely obtained at common outpatient clinics or on admission at most hospitals, could enable timely application of advanced medical examinations and referral to higher level medical institutions, only to appropriate patients.

Although some prediction models have been reported for IE, each of these models has only been applied to one individual bacterium, such as Staphylococcus aureus,^7–9 Viridans streptococci,¹⁰ or Enterococcus spp.¹¹ Because these models are obviously limited to a single bacterial species, they can only be used after identification of causative bacteria by blood culture. To the best of our knowledge, no prediction models for IE exist, except the Duke criteria and the m-DC, for patients with UF.¹²

We herein report the development of a prediction model for IE among patients with UF, using only simple indicators which can be obtained without advanced medical equipment.

Materials and Methods

Study Design and Participants

The present study was a single-center case–control study. Among patients aged ≥20 years at Saga University Hospital from September 2007 to August 2017, patients with International Statistical Classification of Diseases and Related Health Problems-10 (ICD-10) code I-330, which indicates a diagnosis of IE, were identified. Among patients with I-330, those who did not show a fever ≥37 °C before admission, who were referred for valve surgery after medical treatment for IE at the previous hospital, who had nosocomial onset, and who were not admitted to hospital were excluded. Of the remaining patients, those who were diagnosed as “definite” by m-DC as gold standard of diagnosis of IE were allocated to the IE group.

Subsequently, among patients aged ≥20 years at Saga University Hospital from January 2015 to December 2017, those with ICD-10 code R-50-9, which indicates UF, were identified. Among patients with R-50-9, patients without a fever ≥37 °C before admission, patients whose cause of fever was identified at the outpatient department, and patients who were not admitted to hospital were excluded. Of the remaining patients, those with “non-definite” by m-DC were allocated to the UF group. Although patients diagnosed as definite by m-DC among patients with ICD-10 code R-50-9 were principally allocated to the IE group, patients with other definitive diagnoses separate from IE were excluded. The final diagnoses of patients in the UF group were determined according to the ICD-10 based on the diagnoses made and documented in medical records by attending doctors.

The present study was registered at https://www.umin.ac.jp. The design was assessed using the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis, also known as TRIPOD.¹³

Setting

Saga University Hospital is the only university hospital in Saga Prefecture, Kyushu, Japan, which has fully equipped outpatient clinics and 604 inpatient beds across 29 clinical departments, including the departments of general medicine, cardiology, and cardiovascular surgery. The hospital provides high-level medical services mainly for patients in acute disease phases. The Division of Infectious Disease/Prevention and Control in our hospital usually supervises the diagnosis of infectious diseases, including IE, and chooses appropriate antibiotics and treatment durations for all patients with positive blood culture results, as well as when other departments request consultations.

Data Sources

Four physicians from the department of general medicine (GP) at our hospital reviewed medical records and referral letters from previous doctors to collect data of potentially eligible patients. Survey items are shown in Supplementary Table 1. Prolonged antimicrobial use for chronic diseases was excluded from the definition of “antibiotic administration before admission.” The diagnosis of disordered consciousness was made when patients had a Glasgow Coma Scale score of ≤14 or when people who were around patients on a daily basis felt that something was wrong about the behavior of patients. Presence of pleural effusion or pulmonary edema on chest X-ray or chest CT on admission was determined by a radiology specialist. If an interpretation by a radiologist was not available, it was determined by two GPs. When the interpretations of the two GPs were in agreement, pleural effusion or pulmonary edema were determined as “present.” When the interpretations were not in agreement, a final conclusion was reached by discussion between the two GPs. Disseminated infections included pyogenic spondylitis, pyogenic arthritis, spinal epidural abscess, deep-seated abscess, and mycotic aneurysm. Disseminated intravascular coagulation (DIC) was diagnosed according to the DIC scoring system of the Japanese Association for Acute Medicine.¹⁴

Data Analysis

Missing values were handled as “no abnormalities” for categorical variables and were excluded from the analysis for continuous variables. Categorical variables are expressed as percentages and were compared using the χ² test. Continuous variables are expressed as median and interquartile range, and were compared using the Mann–Whitney U-test. Statistical significance was set at p<0.05. In determining covariates of multivariate logistic regression analysis, we excluded items which could not be determined on admission, such as mortality and duration of the hospital stay. We also excluded every item in the m-DC, which we used as the gold standard of the diagnosis of IE, and items with available data less than 50. In light of collinearity, candidate variables that showed significance with a univariate analysis and a low correlation coefficient (in case of a Spearman’s r value>0.7 or a Kendall’s r value>0.7 between two items, one factor was selected, and the others were removed) were selected. A backward selection method was used for multivariate logistic regression analysis.

Using the logistic regression model, the predictive probability of definite IE according to the m-DC were calculated. The cut-off values were set to the maximum sum of sensitivity and specificity. The predictive performance of the model was assessed using the area under the curve (AUC) derived from the same cohort. Model calibration was assessed using the shrinkage coefficient¹⁵ and the Hosmer–Lemeshow test. SPSS Statistics (version 25; IBM Corp., Armonk, NY, USA) was used for statistical analysis.

Ethical Consideration

The study conformed to Ethical Guidelines for Medical and Health Research Involving Human Subjects in Japan. The ethics committee of Saga University Hospital approved the study (file number: 2020-05-R-04), which was conducted in accordance with the guidelines of the 1975 Declaration of Helsinki. All subjects, who were informed about the purpose of the study via the homepage of clinical research center of Saga University Hospital, provided consent as per the comprehensive agreement method, and patient anonymity was protected.

Results

Enrollment and Allocation of Study Participants

Enrollment of participants to the present study and allocation to either the IE group or the UF group are shown in Figure 1. Thirty-three of 108 patients with code I-330, who were without a fever ≥37 °C (n=10), with referral for valve surgery after medical treatment for IE at the previous hospital (n=7), with nosocomial onset (n=14), and who were not admitted to hospital (n=2), were excluded. Of the remaining 75 patients, 54 patients were diagnosed with definite IE according to the m-DC and were allocated to the IE group. Thirty-one of 122 patients with code R-50-9, who were without a fever ≥37 °C before admission (n=4), whose cause of fever was identified at the outpatient department (n=27), and who were not admitted to hospital (n=0), were excluded. Of the remaining 91 patients, six patients were diagnosed with definite IE according to the m-DC, five of whom were allocated to the IE group. One patient was excluded due to a final diagnosis of adult-onset Still’s disease. As a result, there were 59 patients in the IE group and 85 patients in the UF group.

Figure 1 Enrollment and allocation flow diagram. Thirty-three of 108 patients had International Statistical Classification of Diseases and Related Health Problems-10 (ICD-10) code I-330. Among them, patients without a fever ≥37 °C (n=10), referred for valve surgery after medical treatment for infective endocarditis (IE) at the previous hospital (n=7), with nosocomial onset (n=14), and who were not admitted to hospital (n=2) were excluded. Of the remaining 75 patients, 54 patients were diagnosed with definite IE according to the modified Duke criteria and were allocated to the IE group. Thirty-one of 122 patients had ICD-10 code R-50-9. Among them, patients without a fever ≥37 °C before admission (n=4), whose cause of fever was identified at the outpatient department (n=27), and who were not admitted to hospital (n=0) were excluded. Of the remaining 91 patients, six patients were diagnosed with definite IE according to the modified Duke criteria. Five patients were allocated to the IE group, and one patient was excluded with a final diagnosis of adult-onset Still’s disease. As a result, 59 patients and 85 patients were enrolled in the IE group and the undiagnosed fever group, respectively.

Patient Characteristics

Patients’ characteristics are shown in Table 1. The following survey items were more frequently observed in the IE group: dyspnea (IE: 20% vs UF: 6%, p=0.012); neurological symptoms (34% vs 9%, p<0.001), including disordered consciousness, paralysis, or weakness within 2 months before admission; long duration of hospital stay (41 days vs 19 days, p<0.001); transfer by ambulance (51% vs 1%, p<0.001); periodontitis (30% vs 5%, p=0.027); previous IE (5% vs 0%, p=0.041); prosthetic valve (19% vs 0%, p<0.001); 30-day mortality (17% vs 5%, p=0.030); in-hospital mortality (17% vs 6%, p=0.044). The UF group had a higher percentage of patients who were administered antibiotics before admission (56% vs 77%, p=0.030) and patients with joint or back pain within 2 months before admission (17% vs 35%, p=0.008). As for vital signs, the IE group had a higher pulse rate (102 bpm vs 90 bpm, p=0.004) and respiratory rate (23 breaths/min vs 18 breaths/min, p<0.001), with higher systemic inflammatory response syndrome¹⁶ (2 vs 1, p<0.001) and quick sequential organ failure assessment¹⁷ (1 vs 0, p<0.001) scores. As for physical examinations, the IE group had a greater percentage of patients with cardiac murmur (53% vs 9%, p<0.001). No difference was observed in the percentage of patients with limb edema (22% vs 15%, p=0.301). The execution rate of each investigation, breakdown of final diagnoses in the UF group, and causative bacteria in the IE group are shown in Supplementary Tables 2–4, respectively.

Table 1 Univariate Analysis of Patient Characteristics

Laboratory Findings on Admission to Our Hospital

Laboratory findings on admission are shown in Table 2. White blood cell count (IE: 13.5×10³/µL vs UF: 10.5×10³/µL, p=0.004), neutrophil count (89.6% vs 80.4%, p<0.001), D-dimer concentration (8.2 μg/mL vs 3.3 μg/mL, p=0.003), total bilirubin concentration (0.9 mg/dL vs 0.6 mg/dL, p=0.001), lactate dehydrogenase concentration (337 IU/L vs 220 IU/L, p<0.001), blood urea nitrogen concentration (24.1 mg/dL vs 16.0 mg/dL, p<0.001), and serum creatinine concentration (1.1 mg/dL vs 0.8 mg/dL, p=0.012) were higher in the IE group compared with the UF group. Platelet count (11.7×10⁴/µL vs 27.9×10⁴/µL, p<0.001) and serum albumin concentration (2.5 g/dL vs 2.8 g/dL, p=0.033) were lower in the IE group compared with the UF group. Additionally, the frequency of DIC (37% vs 12%, p=0.001) was higher in the IE group compared with the UF group.

Table 2 Univariate Analysis of Laboratory Findings

Imaging Studies

The results of imaging studies are shown in Table 3. Although there was no difference in the morbidity of total valvular regurgitation, mitral regurgitation (IE: 59% vs UF: 40%, p=0.017) was higher in the IE group, and tricuspid regurgitation (29% vs 51%, p=0.029) and pulmonary regurgitation (7% vs 26%, p=0.007) were higher in the UF group. No differences were observed in the percentage of patients with an ejection fraction ≤50% within 14 days after admission (15% vs 6%, p=0.083) and metastatic infection (14% vs 7%, p=0.239). Pulmonary edema (29% vs 1%, p<0.001) and pleural effusion (64% vs 29%, p<0.001) on admission were higher in the IE group. Chest X-ray or CT findings of 87 patients (60.4%), specifically 37 of 59 patients (62.7%) in the IE group and 50 of 85 patients (58.8%) in the UF group, were interpreted by a radiologist. Five patients underwent neither chest X-ray nor CT. Two GPs interpreted the chest X-rays of the remaining 52 patients with concordance rates for pleural effusion and pulmonary edema of 90.4% and 84.6%, respectively.

Table 3 Univariate Analysis of Imaging Studies

Multivariate Logistic Regression Model

The results of the multivariate logistic regression analysis are shown in Table 4. Transfer by ambulance (odds ratio [OR] 59.1, 95% confidence interval [CI] 3.78–922.52), cardiac murmur (OR 17.2, 95% CI 4.49–65.81), pleural effusion (OR 4.2, 95% CI 1.33–13.39), neutrophil count (OR 1.1, 95% CI 1.03–1.16), and platelet count (OR 0.9, 95% CI 0.91–0.98) were significant factors for the prediction of IE.

Table 4 The Results of the Multivariate Logistic Regression Analysis

Performance of the Predictive Model

The detailed prediction model is shown in online Supplementary Appendix 1. The AUC was 0.893 (95% CI 0.828–0.959: Figure 2). The model was well calibrated using the Hosmer–Lemeshow test (p=0.440) with a shrinkage coefficient of 0.635 (Figure 3). The stratum-specific likelihood ratio was 0.2–50.4, increasing as the score increased (Table 5). The cut-off value of the prediction model was −1.1, with a sensitivity of 84.7% and a specificity of 84.7%.

Table 5 Stratified Likelihood Ratio of Our Clinical Prediction Model

Figure 2 Receiver operating characteristic curve and AUC. The predictive performance of the model was assessed by the area under the curve (AUC) derived from the same cohort. The AUC was 0.893 (95% confidence interval 0.828–0.959).

Figure 3 Predictive and observed rates of IE in quadrisect groups according to scores in the internal validation cohort. No difference was found between the expected rate derived from the prediction model and the actual ratio observed in the IE group in each of the four groups with accurate calibration.

Discussion

IE can be misdiagnosed as fever of unknown origin, especially at average hospitals or outpatient clinics that are unequipped with advanced medical modalities, such as MRI, CT, or TEE. Although the m-DC requires such investigations to be performed,⁴ it could be unrealistic to apply expensive and possibly invasive medical modalities to all patients with UF. Therefore, it is of profound significance to select patients with a high probability of IE using indicators that are easy to obtain in common practice, making it possible to choose appropriate patients to undergo such examinations or to be referred to advanced medical institutions. We developed a prediction model for IE among patients with UF using five simple indicators, transfer by ambulance, cardiac murmur, pleural effusion, neutrophil count, and platelet count. These indicators can be obtained at common hospitals and clinics, without requiring advanced medical equipment or invasive examinations. Furthermore, our model could identify or exclude definite IE diagnosed according to the m-DC.

In the present study, transfer by ambulance was one of the predictors of IE. However, it could be inappropriate to depreciate patients visiting without ambulance because this is one of the factors that delays IE diagnosis.¹⁸ In that respect, our model could identify patients with the risk of IE among those who visit hospital without ambulance, because our model was proved to be sufficiently accurate using the same subjects, in which approximately half of them visited without ambulance. Furthermore, a body temperature ≤38 °C and a serum C-reactive protein (CRP) concentration <10 mg/dL also delay IE diagnosis.¹⁸ Our prediction model can identify patients with such delaying factors because our patients had a fever ≥37 °C before admission, with 44% of patients in the IE group showing a serum CRP concentration <10 mg/dL. A similar degree of accuracy was observed using these same subjects.

Even using modernized antibiotics, mortality rates from IE are as high as 11%–30%.^19–21 Furthermore, administration of antibiotics before obtaining blood cultures could cause negative results, which could delay diagnosis.^6,18 Using our prediction model before administration of antibiotics could reduce inappropriate administration of antibiotics prior to obtaining blood cultures by alerting clinicians of a possible IE diagnosis.

One drawback of our study is that IE on prosthetic valves was diagnosed using the m-DC. When diagnosing IE on prosthetic valves, the sensitivity of the m-DC was as low as 50%,²² which could be improved by combined use of ¹⁸F-fluorodeoxyglucose positron emission tomography with CT or single-photon emission CT.^23,24 However, application of these modalities is unrealistic to diagnose IE in Japan due to Japanese Health Insurance system. Thus, further study is required to determine whether our prediction model is useful for IE on prosthetic valves.

The present study had several limitations. Because the annual incidence of IE was low,²⁵ the number of samples that were able to be collected in a single-center pilot case–control study such as the present study was limited. The use of ambulance, which was one of the predictors of IE in the present study, might have differed among regions, countries, or health care services. Therefore, multi-center studies are required to clarify whether the factor of transfer by ambulance can be widely used. Finally, external validation is preferable because the AUC of our prediction model was derived using the same cohort as was used to develop the model.

Conclusion

We developed a prediction model for infective endocarditis in patients with undiagnosed fever using only easily obtainable data, transfer by ambulance, cardiac murmur, pleural effusion, neutrophil count, and platelet count. Our prediction model could make early detection of patients with IE possible, resulting in proper treatments and reducing inappropriate practices, such as administration of antibiotics prior to obtaining blood cultures.