Obstructive Sleep Apnea Screening with a 4-Item Instrument, Named GOAL Questionnaire: Development, Validation and Comparative Study with No-Apnea, STOP-Bang, and NoSAS

Introduction

Obstructive sleep apnea (OSA) is characterized by frequent partial or complete upper airway collapse during sleep, resulting in intermittent hypoxemia and sleep fragmentation.¹ There is growing evidence that OSA plays a key role in the pathogenesis of cardiovascular and metabolic diseases, as well as a clear association with increased overall mortality and medical costs.^2,3 OSA is very prevalent disease,^4–6 being that most recent estimates have suggested that nearly a billion of people may be affected, with about 425 million individuals, aged 30 to 69, suffering from moderate to severe OSA, worldwide.⁷

The gold standard for the diagnosis of OSA is an attended in-lab overnight polysomnography (PSG) test.¹ However, this costly method is not readily available to a large number of patients with suspected OSA, especially in regions with limited economic resources.^1,8 Due to the high frequency of OSA, there is a significant cost in evaluating all subjects with suspected OSA, leading to long waiting times for testing.⁹ In this context, portable diagnostic methods have been extensively implemented as an alternative for the diagnosis of OSA in those adults exhibiting a high pretest probability and absence of significant comorbidity.¹

The various screening instruments that have merged over the years have been generally predicated on the clinical, demographic and anthropometric parameters that constitute the major risk factors for OSA.⁸ The STOP-Bang questionnaire is an 8-item acronym that can be easily completed.¹⁰ It was developed and later validated in surgical subjects having a high prevalence of OSA_≥5: 73% (derivation cohort) and 69% (validation cohort).¹⁰ Subsequently, it was widely validated in several different settings: sleep clinic patients, surgical patients, and general population.^11–16 Conversely, NoSAS score was derived and validated in population-based cohorts (HypnoLaus and EPISONO, respectively).¹⁷ Afterwards, this score was also validated in different settings, reporting adequate performance as screening model for OSA: in a multiethnic Asian cohort, in a hospital-based sample, in depressive subjects, and in a sleep clinic.^18–22

The No-Apnea²³ tool is a newly developed and validated screening instrument that includes only two objective parameters: neck circumference (NC) and age, with a total score ranging from 0 to 9 (cutoff point ≥3 classifies patients at high risk for OSA). In derivation and validation cohorts, No-Apnea showed adequate performance in OSA screening, with a discriminatory ability that was similar and indistinguishable from the performance of either STOP-Bang or NoSAS.²³ Although No-Apnea has been subsequently validated in selected clinical populations,^24–26 and displayed adequate predictive performance, some points deserve to be highlighted: i) absence of subjective variables may result in more difficult applicability, especially in regions with a lower prevalence of OSA, such as in primary care; ii) presence of age ≥55 years, already sufficient to classify individuals at high risk for the diagnosis of OSA, possibly limits its use in older populations; and iii) because it contains different scores for each parameter, it may be difficult to adopt and implement, particularly in general clinical practice settings.

Based on such concerns, we aimed to develop a practical and concise questionnaire that would include both objective and subjective variables, with yes-or-no dichotomous answers. To this effect, we designed the present study into two parts: i) derivation and validation of a proposed tool for screening OSA in adults, and ii) comparison with three other widely validated OSA screening instruments: No-Apnea score, STOP-Bang questionnaire, and NoSAS score.

Methods

Study Design and Patient Selection

This was a cross-sectional study, comprising the period from January 2017 to June 2019, including adults which were consecutively referred for PSG evaluation due to suspected sleep disordered breathing by their attending physicians. Then, all subjects were grouped into two separate cohorts: derivation (from January 2017 to February 2018) and validation (from May 2018 to June 2019). Inclusion criteria consisted of individuals of both genders, aged ≥18 years and with suspected of OSA, while exclusion criteria were: previously diagnosed OSA, use of home sleep study for diagnosis, incomplete clinical data and technically inadequate PSG. In case the same individual underwent more than one PSG, only the test with the longest total sleep time was included. All sleep tests were conducted in a Brazilian single-center located in the city of Rio de Janeiro: SLEEP – Laboratório de Estudo dos Distúrbios do Sono.

On the evening of PSG, clinical, demographic and anthropometric data were systematically obtained from all participants by trained sleep laboratory technicians: gender, age, body mass index (BMI), NC, self-reported comorbidities (hypertension and diabetes mellitus), OSA-related symptoms (snoring, observed apnea, gasping/choking, and tiredness), and Epworth Sleepiness Scale (ESS).²⁷ The ESS is an 8-item questionnaire (each item is scored from 0 to 3 points, with a final score from 0 to 24 points), which subjectively measures excessive daytime sleepiness by a score ≥11 points.²⁷ BMI was calculated dividing the weight in kilograms by the square of the height in meters (kg/m²), while NC (cm) was measured using a tape measure, with its upper edge placed below the laryngeal prominence and applied perpendicularly along the neck axis.

Ethical Considerations

The study protocol (No. 1.764.165) was approved by the Research Ethics Committee of the Federal University of Rio de Janeiro (UFRJ) and adhered to standard previously set by the Declaration of Helsinki. All participants provided written informed consent, being that anonymity of all recruited individuals was guaranteed throughout the study process.

Screening Instruments

The No-Apnea (final score from 0 to 9, high risk with 3 or more points) contains two objective parameters: NC is scored in three values: 1 (37.0–39.9 cm), 3 (40.0–42.9 cm), and 6 (≥43.0 cm), while age is scored as follows: 1 (35–44 years), 2 (45–54 years), and 3 (≥55 years).²³ The STOP-Bang (final score from 0 to 8, high risk with 3 or more points) consists of 8 yes-or-no questions (1 point for each affirmative answer): loud snoring, tiredness, observed apnea, hypertension, BMI > 35 kg/m², age >50 years, NC > 40 cm, and male gender.¹⁰ The NoSAS allocates 4 points for having a NC >40 cm, 3 points for having a BMI of 25–29 kg/m² or 5 points for having a BMI ≥ 30 kg/m², 2 points for habitual snoring, 4 points for age >55 years, and 2 points for men; this instrument is considered positive with a score ≥8 points (from 0 to 17 points).¹⁷

Sleep Studies

All subjects underwent an attended, in-lab PSG (EMBLA^® S7000, Embla Systems, Inc., Broomfield, Colorado, United States) consisting of continuous monitoring of electroencephalogram, electrooculogram, electromyogram (chin and legs), electrocardiogram, airflow, thoracic and abdominal impedance belts, oxygen saturation, snoring microphone and sensors for body position. Polysomnographic records were manually interpreted by two board-certified sleep physicians, according to a guideline previously published in 2012 by the American Academy of Sleep Medicine (AASM),²⁸ which were blinded to the values of all screening instruments collected prior to PSG. Apneas were classified from a drop ≥90% of baseline airflow lasting at least 10 s, while hypopneas were classified from a ≥30% pre-event drop over ≥10 s associated with desaturation of oxygen ≥3% or an arousal.²⁸ The AHI was calculated as the number of apnea plus hypopnea/total sleep time (in hours). Polysomnographic diagnosis of OSA was based on apnea/hypopnea index (AHI) ≥5.0/h and its severity was classified as follows: ≥5.0/h as any OSA (OSA_≥5), ≥15.0/h as moderate/severe OSA (OSA_≥15), and ≥30.0/h as severe OSA (OSA_≥30).

Statistical Analysis

Data analysis was carried out using SPSS for Windows (version 21.0; SPSS; Chicago, IL, United States). Results are presented as mean ± standard deviation for continuous variables and as frequency (n) with percentage (%) for categorical variables. The groups were compared using the chi-square test for categorical variables, while Student’s t-test and univariate analysis of variance (ANOVA) were used for numerical variables. The modeling of the proposed instrument was initially based on 10 variables often associated with OSA diagnosis: male gender, age ≥50 years, NC ≥40 cm, BMI ≥30 kg/m², loud snoring, observed apnea, hypertension, diabetes mellitus, tiredness, and gasping/choking. The chosen cutoff points for age and BMI were based on the highest odds ratio (OR) obtained for OSA_≥15 diagnosis: age ≥50 years (OR: 1.911; 95% confidence interval [CI]: 1.668–2.189) versus age ≥55 years (OR: 1.817; 95% CI: 1.566–2.108) and BMI ≥ 30 kg/m² (OR: 2.563; 95% CI: 2.243–2.928) versus BMI ≥ 35 kg/m² (OR: 2.088; 95% CI: 1.826–2.388). Similarly, we used loud snoring and not the presence of snoring by obtaining a higher OR for OSA_≥15 diagnosis: OR: 5.454 (95% CI: 3.972–7.490) versus 3.801 (95% CI: 3.300–4.377). The NC ≥ 40 cm has been chosen as this is the threshold used in the STOP-Bang and NoSAS instruments.^10,17 Firstly, these 10 chosen variables were evaluated by univariate analyses, where the outcome was an AHI ≥ 15.0/h, a conventionally accepted cutoff point for classifying clinically relevant OSA. Sequentially, clinically relevant variables (p < 0.10 in the univariate analysis) were included in a multivariate logistic regression model with AHI ≥ 15.0/h being the dependent variable. Performance of the newly developed instrument was calculated as follows: i) discrimination by the Receiver Operating Characteristic (ROC) curves and area under the curve (AUC); ii) calibration assessed by the Hosmer–Lemeshow test (p < 0.05 was considered as poor calibration); iii) 2x2 contingency tables (sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]); and iv) correlation through the Spearman coefficient (r_s). As Hosmer–Lemeshow test is very sensitive to sample size, we chose a smaller subset of randomly selected patients (n = 1000) to evaluate calibration. An AUC > 0.7 was considered as clinically significant, being that AUCs were compared using a previously described algorithm.²⁹ All predictive parameters were calculated at three AHI thresholds (5.0/h, 15.0/h, and 30.0/h) and reported with their respective 95% CIs. A two-tailed p-value <0.05 was considered as statistically significant.

Results

The flowchart of the study is showed in Figure 1, being that 7377 subjects were consecutively allocated into two independent cohorts: one for derivation (n = 3771) and one for validation (n = 3606). According to Table 1, no clinical or polysomnographic parameter showed a statistically significant difference between derivation and validation cohorts, except for total sleep time (p = 0.047) and arousal index (p = 0.031). Prevalence of OSA_≥5, OSA_≥15, and OSA_≥30 showed no statistically significant differences between derivation and validation cohorts: 79.1% versus 78.8% (p = 0.753), 57.7% versus 57.0% (p = 0.604), and 37.8% versus 35.7% (p = 0.070); respectively.

Table 1 Patient Characteristics

Figure 1 The flowchart of the patients. Abbreviations: OSA, obstructive sleep apnea; PSG, polysomnography.

Modeling

Table 2 shows the seven independent predictive parameters for screening for OSA_≥15. Diabetes mellitus and gasping/choking, although initially tested as possible OSA predictors, were subsequently excluded because they are not independent variables for OSA_≥15 diagnosis. As can be seen in Table 2, these seven parameters were ranked according to adjusted OR: the main parameter was male gender, followed by age ≥50 years, loud snoring, BMI ≥ 30 kg/m², NC ≥ 40 cm, observed apnea, and hypertension.

Table 2 Univariate and Multivariate Analyses of Clinical Parameters According to AHI ≥ 15.0/h (Derivation Cohort; n = 3771)

From these data, several possible models were tested using the main clinical parameters previously found. Figure 2 summarizes the discrimination of possible models containing from 3 to 7 parameters, which exemplifies why a 4-item model was ultimately chosen, since discriminatory power did not significantly increase with the addition of a 5th (NC ≥ 40 cm), 6th (observed apnea) or 7th variable (hypertension). This 4-item instrument, because it contains only yes-or-no dichotomous answers, reaches a final score of 0–4 points (Table 3), being later named by the acronym GOAL (gender, obesity, age, and loud snoring) questionnaire.

Table 3 The GOAL Questionnaire

Figure 2 Graphical representation of several models sequentially constructed with the main independent variables for screening of moderate/severe obstructive sleep apnea (OSA≥15). 3-item model: male gender, age ≥50 years, and loud snoring; 4-item model: 3-item model plus body mass index ≥30 kg/m2, 5-item model: 4-item model plus neck circumference ≥40 cm, 6-item model: 5-item model plus observed apnea, and 7-item model: 6-item model plus hypertension. The 3-item model reported discrimination statistically lower than that obtained by 4-item model (p < 0.001). The 4-item model had a similar discrimination when compared to the 5-item, 6-item or 7-item models (p-values: 0.184, 0.070, and 0.086; respectively). Estimates reported as area under the curve (95% confidence interval).

Predictive Parameters

Mean AHI values increased linearly in parallel with increasing GOAL questionnaire scores (from 0 to 4 points): i) derivation cohort: from 3.6 ± 5.7/h to 53.3 ± 26.1/h (p-value for trend <0.001) and ii) validation cohort: from 3.6 ± 5.8/h to 50.8 ± 27.1/h (p-value for trend <0.001); Figure 3. In both datasets, GOAL was positively correlated with AHI: r_s = 0.559 with p < 0.001 (derivation cohort) and r_s = 0.554 with p < 0.001 (validation cohort). In the derivation cohort, GOAL questionnaire showed adequate calibration for screening of OSA_≥5, OSA_≥15, and OSA_≥30 assessed by Hosmer–Lemeshow test: 11.563 (p = 0.172), 5.135 (p = 0.743), and 5.355 (p = 0.719); respectively. Similarly, in the validation cohort, GOAL displayed adequate calibration for screening of OSA_≥5, OSA_≥15, and OSA_≥30 assessed by Hosmer–Lemeshow test: 9.855 (p = 0.197), 3.921 (p = 0.864), and 5.799 (p = 0.670); respectively.

Figure 3 Mean apnea/hypopnea index (AHI) values obtained by polysomnography according to GOAL questionnaire scores (from 0 to 4 points).

Predictive performance of the GOAL questionnaire is shown in Table 4 (derivation cohort). We used a cutoff point ≥2 to classify patients at high risk for OSA. This strategy focused on privileging sensitivity over specificity, aiming at reducing the false-negative rate. The frequency of subjects classified as high risk for OSA through the four screening instruments is shown in Figure 4.

Table 4 Predictive Parameters of the GOAL Questionnaire (Derivation Cohort; n = 3771)

Figure 4 Percentage of individuals assessed as high risk for diagnosis of obstructive sleep apnea by four screening instruments: GOAL questionnaire, No-Apnea score, STOP-Bang questionnaire, and NoSAS score.

Table 5 compares GOAL questionnaire with three other previously validated tools: No-Apnea, STOP-Bang, and NoSAS. In the derivation cohort, for screening of OSA_≥5, OSA_≥15, and OSA_≥30, GOAL questionnaire presented sensitivities ranging from 83.3% to 94.0% and specificities ranging from 62.4% to 38.5%. In the validation cohort, for screening of OSA_≥5, OSA_≥15, and OSA_≥30, GOAL questionnaire showed sensitivity values ranging from 83.7% to 94.2% and specificity values ranging from 63.4% to 37.7%. In both cohorts, for diagnosis of OSA_≥5, OSA_≥15, and OSA_≥30, STOP-Bang always presented the highest sensitivity, while NoSAS always presented the highest specificity. In both datasets and at all OSA severity levels, GOAL questionnaire showed high sensitivity. As expected, for all four instruments, as AHI thresholds increased, a corresponding increase in sensitivity and a reduction in specificity occurred.

Table 5 Predictive Parameters of the OSA Screening Models

Comparing Discriminatory Ability

Figure 5 shows the discrimination obtained by the four screening instruments: in both cohorts and at all OSA severity levels, there were no statistically significant differences in the discriminatory ability of the GOAL questionnaire when compared to No-Apnea, STOP-Bang or NoSAS in screening of OSA_≥5, OSA_≥15, and OSA_≥30.

Figure 5 Discriminatory ability, reported as area under the curve (95% confidence interval), of GOAL questionnaire, No-Apnea score, STOP-Bang questionnaire, and NoSAS score for screening of obstructive sleep apnea (OSA) assessed by an apnea/hypopnea index (AHI) ≥5.0/h (OSA≥5), ≥15.0/h (OSA≥15), and ≥30.0/h (OSA≥30). In the derivation cohort, GOAL questionnaire had similar discrimination to No-Apnea, STOP-Bang, and NoSAS for predicting OSA≥5 (p-values: 0.476, 0.928, and 0.532; respectively), OSA≥15 (p-values: 0.404, 0.778, and 0.354; respectively), and OSA≥30 (p-values: 0.557, 0.123, and 0.674; respectively). In the validation cohort, GOAL questionnaire showed equivalent discrimination to No-Apnea, STOP-Bang, and NoSAS for predicting OSA≥5 (p-values: 0.593, 0.786, and 0.115; respectively); OSA≥15 (p-values: 0.927, 0.465, and 0.651; respectively), and OSA≥30 (p-values: 0.936, 0.293, and 0.749; respectively).

Discussion

Current findings show that in a sleep-lab setting, a simple and practical 4-item model, which we have denominated GOAL questionnaire (male gender, obesity with BMI ≥30 kg/m^2, age ≥50 years, and loud snoring), achieves adequate and reproducible predictive performance for OSA screening at all severity levels. Furthermore, its discriminatory ability was similar from that of three previously validated and widely used screening instruments for OSA: No-Apnea score, STOP-Bang questionnaire, and NoSAS score. Moreover, obtaining score ≥2 versus <2 points to rank, respectively, at high or low risk allowed an increase in specificity values compared to No-Apnea, while concurrently preserving a relatively high sensitivity, especially at most severe levels.

Sensitivity and specificity of a screening model are usually inversely related and high sensitivity often comes at the expense of specificity.^8,30 In the context of OSA, which is a very prevalent disease and often associated with significant morbidity, it may be more important that the screening method has high sensitivity, with consequent reduction in the false-negative rate.^8,30 As expected, all four screening instruments (GOAL, No-Apnea, STOP-Bang, and NoSAS) showed increased sensitivity in parallel with increasing AHI thresholds (from 5.0/h to 30.0/h). On the other hand, specificity decreased in the most severe forms of OSA, which can result in a considerable number of false positives.

Although our newly developed questionnaire contains twice as many parameters as No-Apnea, it may be easier to respond, score and interpret, since its construct is predicated on yes-or-no answers leading to easy calculation of the cumulative score and simple attribution of risk. In contrast, No-Apnea contains several sub-items being scored from 0 to 9 points. Another noteworthy fact is that No-Apnea derivation and validation study²³ was designed with retrospectively recruited subjects, while the present study was conducted with two independent cohorts of prospectively enrolled subjects. In addition, logistic regression employed for No-Apnea model development²³ was performed with binary outcome (AHI ≥ 5.0/h), while in the development of the GOAL questionnaire we employed AHI ≥ 15.0/h, a threshold often implicated as a clinically relevant demarcation for OSA. Our recently proposed instrument contains only one subjective variable (loud snoring), similar to NoSAS (presence of snoring), but well below the STOP-Bang which has four subjective variables (loud snoring, observed apnea, tiredness, and hypertension). Using a limited number of subjective parameters possibly could result in greater applicability of the instrument and less possibility of bias.

Another feature of the current instrument is that by increasing its scores (from 0 to 4 points) there was a linear increase in AHI obtained by PSG. This property may further translate into a valuable tool for health professionals in defining the level of risk and relative certainty when attempting to identify those patients with a higher or lower probability of having a diagnosis of OSA. The risk escalation characteristic has also been previously described with the STOP-Bang questionnaire, whereby increasing values obtained from STOP-Bang (from 0 to 8 points) were associated with an increased likelihood of having OSA.³¹

The four clinical parameters (gender, BMI, age, snoring) employed for the elaboration of our screening instrument are well-established predictive factors for OSA. Regarding clinical symptomatology, men with OSA usually have typical symptoms such as snoring and observed apnea, while women with OSA often report symptoms that are considered atypical, such as insomnia, morning headache, and fatigue.^32,33 Based on polysomnographic findings, women have a lower prevalence of OSA than men, but with a clear increase in frequency after menopause.^32,33 Unlike No-Apnea,²³ the chosen measure of obesity was BMI over NC: this finding may translate into a better balance of its components between genders, since NC is a variable that is arguably higher in men than in women.^32–34 OSA prevalence increases with age, with some studies reporting a high prevalence of OSA in the elderly.³⁵ The only subjective variable that was chosen in our model was loud snoring, since snoring intensity increases as OSA becomes more severe.³⁶ In addition, this question from the STOP-Bang questionnaire “Do you snore loudly?” has already undergone cross-cultural adaptation to the Portuguese language spoken in Brazil.³⁷

Limitations and Strengths

Our study has some limitations that should be highlighted: the study population was based on sleep-lab referral patients (subjects with a high pretest probability), and therefore the possibility of selection bias is plausible. As the predictive values of a screening model are affected by disease prevalence,^30,38 future studies should be conducted to assess the performance of the GOAL questionnaire in other population groups. In addition, it was conducted in a single-center and did not preferably include individuals belonging to populations with their own distinct anthropometric characteristics (eg, Asian or African populations), thus requiring future additional external validation. Conversely, we should also point out several features of the present study that should strengthen the ability to implement the proposed instrument: it was developed and validated from two very large and representative cohorts, with prospectively recruited participants undergoing the same diagnostic test for OSA (overnight full PSG) and with the same diagnostic criteria recommended by AASM.²⁸

Conclusions

All instruments (GOAL questionnaire, No-Apnea score, STOP-Bang questionnaire and NoSAS score) were found to be adequate instruments for OSA screening at any severity level. The use of these instruments can allow improved allocation of patients into corresponding priorities, thus enabling better prioritization of financial resources. In both cohorts, there was no superiority of one given tool over the other, which shows a possible great practical applicability of the GOAL questionnaire, as it contains only four clinical parameters easily obtained during any evaluation of a patient with suspected OSA. As with any population study, future exploration for other world regions and different clinical settings will be critical for widespread implementation of such simple and concise screening tool.