Development and Validation of a Nomogram for Predicting Obstructive Sleep Apnea in Bariatric Surgery Candidates

Introduction

Obstructive sleep apnea (OSA) is a common sleep-related disorder that is characterized by repeated episodes of partial or complete upper airway obstruction during sleep, resulting in repetitive hypoxemia, sleep fragmentation, morning headaches and excessive daytime sleepiness.^1–3 Long-term complications of OSA include type 2 diabetes mellitus (T2DM), hypertension, cardiovascular disease (atrial fibrillation, heart failure and coronary heart disease), stroke, and sudden-death during sleep.^1,4,5 Among the multifactorial etiologies for OSA development, obesity is a well-known risk factor and predictor, and approximately 70% of patients with diagnosed OSA are obese.^6–8

Recently, bariatric surgery has been increasing in popularity because it achieves sustained weight loss and improves obesity-related complications.⁹ In patients evaluated for bariatric surgery, the prevalence of OSA ranges from 59.9% to 93.5% depending on the demographic characteristics of the participants.¹⁰ Although the prevalence of OSA is high in bariatric surgery candidates, its underdiagnosis is a common issue. A prospective study of 834 patients showed that approximately 40% of bariatric surgery candidates had no previous diagnosis or had not undergone any screening before starting preoperative evaluation.¹¹ Bariatric surgery patients with OSA have greater anesthetic risk, higher risk for perioperative and postoperative complications.^12,13 The underdiagnosis OSA may lead to devastating complications and potentially increase a patient’s morbidity and mortality, suggesting the early diagnosis and management of OSA is extremely important. Thus, the routine clinical screening for OSA should be recommended in the bariatric population, although some patients with OSA may be asymptomatic.^14,15

Currently, nocturnal polysomnography (PSG) remains the gold standard for diagnosing OSA but it is expensive, time-consuming, and uncomfortable for patients, which makes some patients refuse to undergo PSG. Therefore, this diagnostic tool is difficult to be widely available in patients with suspected OSA, especially in economically underdeveloped areas. Although several questionnaires (Berlin Questionnaire, STOP-Bang, GOAL and Fatigue Severity Scale (FSS)) are available for detecting patients with high risk of OSA,^16,17 their application value in patients who undergo bariatric surgery remains unclear, and further validation is warranted. A prospective study of 411 bariatric surgery patients showed No-Apnea, STOP-Bang questionnaire and NoSAS score were useful tools for screening of OSA in morbidly obese patients, but their accuracy was less than 80%.¹⁸ In another study with 425 bariatric surgery patients, Epworth Sleepiness Scale (ESS) was considered to have no significant predictive value before bariatric surgery.¹⁹ In addition, some study showed there were gender-specific differences in the performance of screening questionnaires (ESS, FSS, STOP-Bang, and NoSAS) in the bariatric population.²⁰ Duarte et al developed a tool of NO-OSAS to identify moderate to severe OSA among bariatric surgery patients, but its performance needs to be further verified in other cohorts.²¹ Therefore, we need to further explore better screening tools to screen for OSA in the bariatric population.

Nomogram is a concise and practical tool for predicting risk by incorporating important predictors to predict clinical events and outcomes,^22,23 and it has been widely used to predict the risk and prognosis of various diseases. The present study aimed to develop and validate a clinical nomogram for accurately predicting the risk of OSA based on preoperative clinical features in bariatric surgery candidates.

Methods

Patient Population

The data of consecutive patients who planned to undergo bariatric surgery were retrospectively collected in our hospital between September 2015 and January 2020. Inclusion criteria included the following: patients aged 18 to 65 years old, BMI≥35 kg/m² or BMI≥27.5 kg/m² combined with inadequately controlled T2DM and at least one other metabolic disease;²⁴ patients agreed to be screened preoperatively by polysomnography. Exclusion criteria were as follows: alcohol or drug abuse, severe depression, other severe diseases contraindicating bariatric surgery, missing or incomplete data, recording failed or recording time less than 300 min. The study was approved by the ethics committee of the First Affiliated Hospital of Jinan University. Furthermore, it was performed in accordance with the principles of the Declaration of Helsinki. Written informed consent from all eligible patients was obtained before bariatric surgery.

Data Collection

Demographic data were collected, including gender, age, weight and height, as well as neck circumference, chest circumference, waist circumference and hip circumference. We evaluated waist-to-hip ratio (WHR), waist-to-height ratio (WtHR) and neck-to-height ratio (NHtR) based on these parameters. In addition, we also collected the data on co-morbidity (fatty liver, hypertension, T2DM, hyperlipidemia, hyperuricemia). Neck circumference was measured at the level of the laryngeal prominence; chest circumference was measured at the level of nipple point; waist circumference was measured at the level of the umbilicus; hip circumference was measured at the horizontal circumference of the most prominent part of the hips backwards. BMI was calculated using a formula that divides weight by height into squares. Habitual snoring was defined as a snoring frequency ≥3 nights per week.²⁵ Fatty liver was diagnosed with color ultrasound. Criteria for the diagnosis of Hypertension were systolic blood pressure≥140 mm Hg or diastolic blood pressure≥90 mm Hg, a history of hypertension or the use of antihypertensive drugs. Criteria for the diagnosis of T2DM was fasting plasma glucose (FPG)≥7.0 mmol/L (126 mg/dL), random plasma glucose≥11.1 mmol/L (200 mg/dL), 2-hour plasma glucose value after a 75-g oral glucose tolerance test (OGTT)≥11.1 mmol/L(200 mg/dL), a history of T2DM or the use of antihyperglycemic drugs. Criteria for the diagnosis of Hyperlipidemia were serum triglycerides above 1.70 mmol/L, serum total cholesterol above 5.7 mmol/L, serum low-density lipoprotein (LDL) cholesterol above 3.76 mmol/L, serum high-density lipoprotein (HDL) cholesterol less than 0.91 mmol/L, or the use of lipid-lowering drugs. Hyperuricemia was defined as serum uric acid above 420 mmol/L in men and above 360 mmol/L in women.

Sleep Study

All included patients underwent full-night PSG (Compumedics E-series, Australia), consisting of oronasal airflow (using dual pressure and thermal sensors), respiratory effort (abdomen and thorax), electrocardiogram, electrooculogram, electroencephalogram, snoring using a tracheal microphone, body position and oxygen saturation (using Nonin finger probe). An experienced sleep specialist scored manually and interpreted polysomnographic recordings in accordance with established guidelines.²⁶ The data of sleep variables were collected from the PSG as follows: total sleep time, sleep efficiency, apnea-hypopnea index (AHI), arousal index, mean oxygen saturation and minimum oxygen saturation.

Apnea was defined as a complete cessation of oronasal airflow for at least 10 seconds.²⁶ Hypopneas were defined as a reduction ≥30% of pre-event baseline in the respiratory signal, for ≥10 seconds in association with either ≥3% arterial oxygen desaturation or an arousal.²² Apnea-hypopnea index (AHI) was defined as the number of apnea and hypopnea events per hour.²⁶ OSA severity was classified according to AHI thresholds: AHI less than 5 events/h was considered normal, AHI 5–15 events/h was considered as mild OSA, AHI 15–30 events/h as moderate OSA and AHI more than 30 events/h as severe OSA.²⁶

Statistical Analysis

Statistical analysis was performed using SPSS 13.0 for Windows (SPSS Inc. Chicago, IL, USA), R software (version 4.0.2 http://www.Rproject.org) and MedCalc version 19.4.0 (Ostend, Belgium). Normally distributed continuous variables are represented by the mean ± standard deviation (SD); non-normal continuous variables are expressed as the median (interquartile range); categorical variables are reported as the number (percentage). Comparison between two groups were examined using Student’s t-test or Mann–Whitney test for continuous data, and Chi-square test or Fisher’s exact test for categorical data.

A permuted randomization number table generated by using the “caret” package of R was applied to randomly assign the patients to the training cohort and validation cohort in a 7:3 ratio. The least absolute shrinkage and selection operator (LASSO) method is used to reduce the dimensionality of data,²⁷ and the optimal predictive features from the training cohort were identified. Features with nonzero coefficients in the LASSO regression model were selected. Then, the features selected in the LASSO regression model were included in the multivariate logistic regression to develop a nomogram. Calibration curves were plotted to assess the calibration of this model, accompanied by the Hosmer-Lemeshow test²⁷ (P > 0.05 is considered as the goodness of calibration); an ideal calibration curve would perfectly fit the 45-degree reference line. The concordance index (C-index) was used to evaluate the discrimination of the nomogram. The nomogram was subjected to bootstrapping validation (1000 bootstrap resamples) to calculate a relatively corrected C-index. Decision curve analysis (DCA) was performed to evaluate the clinical usefulness of the nomogram based on net benefits at different threshold probabilities in the training and validation cohort.²⁸ Area under the curve (AUC) in receiver operating characteristic (ROC) analysis was used to evaluate predictive accuracy. P value of less than 0.05 was considered statistically significance.

Results

Basic Characteristics of Patients

A total of 499 patients who underwent full-night PSG screened for inclusion, 17 patients were subsequently excluded: because of missing or incomplete data, recording failed, recording time less than 300 min. Finally, 482 patients were enrolled to the study, 296 (61.4%) women and 186 (38.6%) men. Of those 482 patients, 342 (71.0%) patients fulfilled the diagnostic criteria for OSA, specifically 126 (26.1%) with mild OSA, 72 (14.9%) with moderate OSA and 144 (44.8%) with severe OSA. In male patients, the prevalence was 88.2% (164/186), compared to 60.1% (178/296) in female subjects (x²=43.57, P=0.000).

Four hundred and eighty-two patients were randomly assigned to either the training cohort (n = 338, for development of the nomogram) or the validation cohorts (n = 144, for validation of the nomogram). There is no significant difference in basic characteristics between the training cohort and the validation cohort (all P>0.05), but compared with non-OSA group, the OSA group had a relatively higher proportion of males, larger demographic variables (weight, height, T2DM, neck circumference, chest circumference, waist circumference and hip circumference, WHR, WtHR, NHtR), and poorer sleep variables. In addition, patients with OSA had a greater prevalence of hypertension, T2DM and hyperlipidemia, as well as a higher proportion of habitual snoring. Table 1 shows the basic characteristics between the two groups.

Table 1 Clinical Characteristics of Patients in the Study

Feature Selection

We used the LASSO logistic regression model to select the most significant predictive features that can help to avoid overfitting. We obtain a more refined model by constructing a penalty function, which makes it compress some regression coefficients. In this study, a linear combination weighted by factor coefficients was used to calculate the risk score, and a cross‐validated error plot was produced (Figure 1A). The model with the optimal lambda values included six variables, and the most parsimonious model with a cross‐validated error within one standard error of the minimum included three variables. The path of each variable coefficient with varying log-transformed lambda values is displayed in Figure 1B. Therefore, the six features with nonzero coefficients were subsequently selected according to LASSO logistic regression and Clinical importance (gender, habitual snoring, T2DM, neck circumference, BMI, age). In addition, twelve variables (hypertension, hyperlipidemia, hyperuricemia, fatty liver, weight, height, chest circumference, waist circumference and hip circumference, WHR, WtHR, NHtR) with zero coefficients were excluded.

Figure 1 Clinical feature selection using the least absolute shrinkage and selection operator (LASSO) logistic regression. (A). Optimal parameter (lambda) selection in the LASSO logistic regression used 10-fold cross-validation via minimum criteria. The black vertical lines were drawn at the optimal values by using the minimum criteria and the one standard error of the minimum criteria (the 1-SE criteria). (B) LASSO coefficient profiles of the 18 features. A coefficient profile plot was produced against the log(lambda) sequence.

Nomogram Construction

The six independent predictors (gender, habitual snoring, T2DM, neck circumference, BMI and age) selected using the LASSO logistic regression were included in multivariate logistic regression analysis to establish the nomogram. Table 2 shows the results of the multivariate logistic regression analysis. A nomogram predicting OSA in bariatric surgery candidates was generated. Each predictor corresponds to a specific score by drawing its line in the Points line. The sum of six predictors is located on the total points line, and a vertical line is drawn downward from the total points axis to the risk of OSA axis. Finally, the predicted probability of OSA is obtained in Figure 2.

Table 2 Prediction Factors for Obstructive Sleep Apnea in Bariatric Surgery Candidates

Figure 2 Nomogram to predict OSA in bariatric surgery candidates. Abbreviations: OSA, obstructive sleep apnea; T2DM, type 2 diabetes mellitus; BMI, body mass index.

Validation of the Model

The nomogram yielded relatively high C-indexes in both the training cohort (0.856, 95% CI: 0.816–0.897) and validation cohort (0.829, 95% CI: 0.76–0.895) which suggested a model with good discrimination. The calibration curve displayed good concordance between prediction and observation in the training and validation cohorts (Figure 3A and B). The Hosmer-Lemeshow test also showed that there was no significant statistical difference in both training cohorts (Chi-square=3.942, p=0.862) and validation cohorts (Chi-square=6.608, p=0.374), suggesting the nomogram was well calibrated. In addition, we also evaluated OSA at AHI cutoffs of 5 events/h, 15 events/h and 30 events/h based on the current nomogram using ROC curve analysis. The results showed that the nomogram also has good diagnostic value at AHI cutoffs of 5 events/h (AUC = 0.856), 15 events/h (AUC = 0.875) and 30 events/h (AUC = 0.55), when AUC of the nomogram is at the optimum cutoff point (Table 3).

Table 3 The Clinical Utility of the Nomogram for Detecting Obstructive Sleep Apnea

Figure 3 Calibration curves for the nomogram in the training cohort (A) and validation cohort (B). The x-axis represents the nomogram-predicted OSA risk. The y-axis is the actual diagnosed OSA. The dotted line represents a perfect prediction by an ideal model. The solid line represents the performance of the nomogram, of which a closer fit to the diagonal dotted line represents a better prediction.

Clinical Use

We evaluated the clinical utility of OSA nomogram using the DCA, as is presented in Figure 4A and B. The results show that when the nomogram-predicted probability of OSA is >9% in training cohorts (Figure 4A) and >17% in validation cohort (Figure 4B); this nomogram provides more net benefit than the treat-all-patients scheme or the treat-none scheme. Thus, the nomogram is clinically useful which enables clinicians to make better clinical decisions.

Figure 4 Decision curve analysis for the OSA nomogram in the training cohort (A) and validation cohort (B). The y-axis measures the net benefit. The blue line represents the nomogram. The black line represents the assumption of intervention-none. The gray line represents the assumption of intervention-all-patients. The decision curve showed that if the threshold probability of a patient and a doctor is >9% (training cohorts) and >17% (validation cohorts), respectively, using this nomogram to predict OSA risk adds more benefit than the treat-all-patients scheme or the treat-none scheme.

Discussion

In the present study, the main finding was that the nomogram was accepted as a suitable and easy-to-use tool to screen OSA in patients undergoing evaluation before bariatric surgery. The nomogram was composed of six variables: gender, habitual snoring, T2DM, neck circumference, BMI and age, which were easily available clinically. In fact, although it is simple to use, our nomogram showed adequate discriminatory ability and good predictive performance. Thus, the nomogram assessment tools are promising for bariatric surgery candidates.

There are many risk factors for OSA, including obesity, male gender, age >50 years, neck circumference >40 cm, hypothyroidism, tonsillar or adenoid hypertrophy, craniofacial abnormalities, and lifestyle factors of smoking and alcohol consumption.^7,29,30 We used LASSO logistic regression to filter out 6 predictors from 18 variables. Three variables (gender, T2DM and BMI) with the P > 0.05 were included in the model, due to considering variables closely associated with OSA. Therefore, the nomograph included 5 objective parameters (gender, T2DM, neck circumference, BMI and age) and 1 subjective parameter (habitual snoring). Of those variables, gender, age, neck circumference and BMI are well-known predictors of OSA. Habitual snoring is a common symptom in patients with OSA, and many questionnaires or models regard it as an important predictor.^31,32 There is sufficient evidence that T2DM is closely related to OSA.^33,34 Shi et al³⁵ established a nomogram that incorporated 6 independent factors (WHR, smoking status, BMI, uric acid, homeostasis model assessment insulin resistance index, and fatty liver) to predict OSA in patients with T2DM. Thus, these predictors are reasonable for the construction of the model.

Validation of the nomogram is important to avoid overfitting and to determine generalizability.²³ In the current study, the nomogram showed favorable discrimination in both the training cohort (C-index=0.856) and validation cohort (C-index=0.829). Furthermore, calibration curves showed optimal agreement between the predicted probabilities and observed outcomes. DCA also indicated that if the threshold probability of a patient was >9% (training cohort) and >17% (validation cohort), using this nomogram added more benefit than the treat-all-patients scheme or the treat-none scheme, which suggested that the nomogram was clinically useful. In addition, the nomogram also provided adequate accuracy for OSA screening at AHI≥15/h or AHI≥30/h (Table 3). Therefore, our nomogram was proved to be robust.

With the ever-increasing prevalence of obesity worldwide, an increase in bariatric surgery in clinical practice has been seen.³⁶ Therefore, it would be important to find a feasible tool to detect OSA preoperatively due to the high prevalence and potentially perioperative risk of OSA. Screening tools such as the Berlin questionnaire, ESS, STOP-Bang and STOP questionnaire (STOP) have been widely used for detecting OSA.¹⁶ However, the optimal tool for the preoperative evaluation of OSA in this specific population is unclear. In a study that included 814 bariatric surgery patients, GOAL, No-Apnea, STOP-Bang, and No-SAS, but not ESS and STOP, enabled satisfactory discrimination as OSA screening tools, but the accuracy of these screening tools is less than 0.8 (AUC<0.8).¹⁷ Nonetheless, a retrospective study of 266 patients who underwent bariatric surgery, the results showed that neither STOP-Bang nor Berlin questionnaires were suitable as screening tools for OSA.³⁷ A prediction model including 13 variables for OSA created by Kolotkin et al was used in bariatric surgery patients, but it was very complicated to use and did have inadequate sensitivity and specificity.³⁸ Xu et al developed and validated a nomogram including 8 variables (age, sex, neck circumference, apolipoprotein B, glucose, insulin, BMI and waist circumference) and showed good accuracy and discrimination,³⁹ but it included some biochemical parameters which made the use of this nomogram more complicated. In the present study, the nomogram performed well and served as an effective screening tool to identify patients with OSA before bariatric surgery because both training and validation cohorts had high accuracy (C-index >0.8). It is worth noting that the primary purpose of the nomogram is to help bariatric surgeons to select those who emerge as high-risk OSA to PSG, but not to replace the standard diagnostic method. More importantly, depending on the nomogram, bariatric surgeons could address such screening issues that the patient refuses to undergo PSG but has a high-risk OSA, thereby reducing the use of costly and time-consuming PSG.

The ultimate purpose of using nomograms is to help clinicians and patients to interpret individual needs for additional intervention or treatment. In this study, in order to prove its clinical usefulness, decision curve analysis was used to evaluate whether decision-making using OSA nomograms would improve patient outcomes. The results showed that the threshold probability of a patient was >9% in training cohort, which meant that all patients can benefit from using this model, but the degree of benefit was different. For example, if the patient would opt for a sleep study if his probability of OSA was 60%, then the net benefit is 0.44, when using the nomogram to make the decision of whether to undergo intervention, with added benefit than the treat-all scheme or the treat-none scheme. Therefore, those patients with a high pretest for OSA should decide to undergo PSG monitoring as soon as possible. Generally speaking, these patients with moderate OSA (AHI 15.0–29.9) and symptomatic or severe OSA (AHI ≥30) should be recommended continuous positive airway pressure (CPAP).⁴⁰ Our model may be difficult to distinguish whether the patient is at risk of moderate or severe OSA. There is still a lack of evidence for decision-making in CAPA treatment through this model. In future research, we can further explore whether to perform CAPA treatment by establishing a model.

This study has several limitations. First, our nomogram was based on a retrospective data collection from a single center and we are unable to collect data regarding ESS and STOP-Bang, which may compromise the generalization of our results. Secondly, there was a lack of external data to validate the model, so whether these findings can be widely used in other populations is unclear. Thirdly, some factors that may be associated with OSA were not considered, such as smoking status, cardiovascular disease and racial differences. Given these limitations, further efforts on prospective and multicenter data collection are encouraged to prove the robustness of nomogram. However, our nomogram can provide a screening tool for bariatric surgeons to evaluate the risk of OSA in bariatric surgery candidates, which is more important for those who refuse to undergo PSG before bariatric surgery. In addition, this novel nomogram helps obese patients to assess the risk of OSA and better make decisions to further refer them to a sleep center for diagnostic PSG.

Conclusions

In conclusion, we developed and validated a novel nomogram with adequate performance and discrimination for predicting OSA in bariatric surgery candidates, regardless of the severity level. The nomogram could serve as a concise and practical tool to help clinicians select bariatric surgery patients with high-risk OSA to PSG. Despite the promising prospect, further prospective and multicenter studies are warranted to validate the robustness of the proposed nomogram.