Development and Validation of a Risk Score for Mild Cognitive Impairment in Individuals with Type 2 Diabetes in China: A Practical Cognitive Prescreening Tool

Introduction

There has been a substantial increase both in the prevalence of diabetes and dementia over the past decades, both of which have become major public health problems. The latest epidemiological evidence shows that there are about 140.9 million adults with diabetes,¹ 15.07 million old people with dementia and 38.77 million old people with mild cognitive impairment (MCI) in China.² A meta-analysis revealed that diabetes is not only an independent risk factor for MCI and dementia but also promotes the transformation from MCI to dementia.³ Diabetes-associated cognitive dysfunction as a brain complication of diabetes has attracted much attention. Cognitive dysfunction can affect many aspects of life including diabetes self-management with potentially serious results.⁴ There is no effective strategy to treat or revert the process of dementia. MCI, which is a transitional stage between cognitively normal aging and dementia, is possibly modifiable.⁵ Therefore, early identification of MCI in patients with diabetes is beneficial to the recovery of cognitive function or take effective measures to delay cognitive decline, as well as self-management of diabetes.

Professional guidelines on diabetes medical care increasingly concern on screening and management of cognitive impairment in geriatric patients with diabetes.^6–9 The American Diabetes Association (ADA) suggests that older patients with diabetes who aged ≥65 years or exhibit problems in self-management should be screened for cognitive function periodically.⁹ Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) are the most commonly recommended cognition assessment tools.^4,9 However, only a small proportion of medical staff had put cognitive screening into daily practice of diabetes care.¹⁰ There are many difficulties hindering cognitive screening, such as the cognitive tests are time-consuming and need corporation of the patients, and the medical staff usually have insufficient time and knowledge.¹⁰ Besides, there is no clear screening recommendation on how to identify patients at high risk of cognitive impairment under 65 years old. Risk prediction models have provided valuable tools for prevention and management of chronic diseases, which can be used to screen high-risk population and target preventive treatment. Therefore, developing a simple and rapid risk score model that can identify individuals at high risk of MCI is critical to early detection of MCI. However, previous studies mainly focused on risk factors or biomarkers associated with MCI in patients with diabetes, and few multivariate risk prediction models for MCI have been developed in T2DM patients.^11,12

The aim of this study was to develop and validate a risk score model to assess the MCI risk in adult patients with T2DM, which can be easily implemented in daily clinical practice. The risk score is expected to be used as a rapid screening tool to identify subjects who are at high risk of MCI and provide rapid risk stratification for individualized management.

Research Design and Methods

Study Population

This cross-sectional study was conducted at a tertiary care hospital in China. The study consecutively recruited 1809 adult inpatients with T2DM from the Department of Endocrinology of Nanjing Drum Tower Hospital from January 2016 to September 2022. Participants inclusion and study flowchart is shown in Supplementary Figure 1. T2DM was diagnosed according to the American Diabetes Association criteria:¹³ glycosylated hemoglobin-type A1c (HbA_1c) ≥6.5% (using a method that is NGSP certified and standardized to the Diabetes Control and Complications Trial assay), or a fasting plasma glucose (FPG) ≥126 mg/dL (7.0 mmol/L), or 2-hour plasma glucose ≥200 mg/dL (11.1 mmol/L) during a 75g oral glucose tolerance test, or in a patient with classic symptoms of hyperglycemia or hyperglycemic crisis, a random plasma glucose ≥200 mg/dL (11.1 mmol/L). In the absence of unequivocal hyperglycemia, results should be confirmed by repeat testing. The inclusion criteria of the participants were as follows: aged ≥20 years; willing and capable to complete the cognitive tests. Exclusion criteria were as follows: (1) those who refused to participant or cannot complete the cognitive tests; (2) patients with severe visual or hearing dysfunction; (3) illiterates; (4) acute cardiovascular or cerebrovascular disease within the previous 3 months, uncontrolled thyroid dysfunction, acute diabetic complications or severe infection, history of severe cerebral trauma, surgery or tumor, or other systemic diseases that could affect cognitive function obviously; (5) severe anxiety, depression or psychiatric disorders; (6) diagnosed dementia; (7) suspected dementia assessed by MoCA; (8) the activities of daily living (ADL) test score ≤60. One thousand three hundred and fifty-two participants completed the cognitive tests. Forty-five patients were further excluded for suspected dementia or the ADL test score ≤60 after tests, and 51 patients were excluded for missing data. One thousand two hundred and fifty-six patients were finally included in the analysis. In compliance with the Helsinki declaration, all participants had signed an informed consent before their participation. The study was approved by the Ethics Committee of Nanjing Drum Tower Hospital.

Clinical Data Collection and Biochemical Measurements

Sociodemographic and detailed clinical information were collected using a standardized questionnaire including age, sex, education years, duration of T2DM, body mass index (BMI), resting blood pressure, smoking habits, alcohol drinking, medications, and self-reported history of severe hypoglycemia. Diabetic complications and comorbidities were identified via the hospitalization records.

Blood samples were collected after an overnight fasting for over 8 hours. FPG was detected using a hexokinase method (TBA-200FR, Tokyo, Japan), HbA_1c was measured by high-performance liquid chromatography (HLC-73G8, Tosoh, Japan), total cholesterol (TC), triglycerides (TG), high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C) were measured by an auto-analyzer (Abbott Laboratories, USA). The results were recorded.

Definitions of Diabetic Complications and Comorbidities

Severe hypoglycemia was defined as episodes that require assistance from another person for treatment.¹⁴ Microvascular disease was defined as presence of at least 1 of the following: neuropathy, retinopathy, and nephropathy. Cerebrovascular disease was defined as presence of at least 1 of the following: ischemic stroke, lacunar cerebral infarction, transient ischemic attack. Cardiovascular disease was defined as presence of at least 1 of the following: coronary heart disease, myocardial infarction, coronary artery bypass graft, percutaneous transluminal coronary angioplasty, and congestive heart failure. Peripheral atherosclerosis was defined as the atherosclerotic disease other than cardiovascular and cerebrovascular diseases.¹⁵

Cognitive Assessments

Cognitive tests were performed in a quiet room to assess participants’ cognition. Global cognitive function was assessed by the Beijing version of MoCA (www.mocatest.org). The MoCA scale which is a 10-minute, 30-point test has been used as a tool to identify individuals with MCI. A MoCA score ≥26 indicates cognitively normal (CN), and <26 demonstrated objective cognitive impairment, the cut-off value of MCI is 19–25, and a MoCA score <19 indicates dementia.¹⁶ One point was added to the total score if a patient had ≤12 years formal education. Independence of daily functioning was assessed with activities of daily living.¹⁷

Diagnostic Criteria of MCI

MCI was diagnosed according to the following criteria recommended by the National Institute on Aging-Alzheimer’s Association Workgroup¹⁸: (1) cognitive concern from patient, an informant or skilled clinicians; (2) objective evidence of cognitive impairment, a MoCA score <26 demonstrated objective cognitive impairment in this study; (3) intact activities of daily living: the ADL test score >60; (4) absence of dementia, included diagnosed dementia and a MoCA score <19.

Statistical Analysis

Data were checked by two researchers and presented as mean ± standard deviation (SD) or median (inter-quartile range, IQR) for continuous variables, and number (proportion) for categorical variables. All eligible participants were randomly divided into derivation and validation sets at 7:3 ratio. Derivation set was used to generate a risk score model, and the validation set was used to validate the risk score. Logistic regression models were used to calculate crude and multivariate-adjusted odds ratios with 95% confidence intervals for predictors of MCI in patients with T2DM. Factors with a P value <0.10 from the univariate logistic analysis were included in an initial multivariate model, and then a backward elimination procedure was used to remove variables with a P value >0.10. The steps of Framingham Study risk score functions were followed for risk score construction.¹⁹ In order to facilitate the application of the risk models in clinical practice, they were converted into integer risk scores in which the continuous variables were stratified. The risk score was developed by substituting the β coefficients of the final prediction model by points.

Discrimination and calibration were used to assess the predictive accuracy of the risk score model both in the derivation and validation set. Discrimination assessed by area under the receiver operating accepted characteristic (ROC) curve (AUC) were used to assess the model’s ability to distinguish between patients with and without MCI. Calibration refers to the agreement between observed and predicted MCI risk and was calculated by Hosmer–Lemeshow χ² goodness-of-fit test. The optimal cut-off point was chosen from the ROC curve and we also selected two cut-off points to achieve sensitivity over 90% or specificity over 90%.²⁰ Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for two cut-off of the risk score in the derivation and validation set. All statistical analyses were performed with SPSS Statistics, version 26.0. Significance level was set at two-tailed P < 0.05.

Results

Participant Characteristics

A total of 1256 (412 with MCI, 32.8%) patients with T2DM (age: 57.5 ± 11.2 years) were finally enrolled for analysis. Totally 880 patients were randomly allocated in the derivation set and 376 patients in the validation set. There were 285 (32.4%) and 127 (33.8%) patients with MCI in the derivation and validation set, respectively. Table 1 summarizes the basic characteristics and potential predictors of the participants. Age (57.7 ± 11.0 years vs 57.1 ± 11.7 years), education years (12.1 ± 3.4 years vs 12.3 ± 3.6 years), gender (male/female: 593/287 vs 236/140), ever smoking or drinking, diabetic and metabolic indexes, diabetic complications and comorbidities between the derivation and validation set had no statistically significant difference (P > 0.05, Table 1), indicating that all variables were comparable between the two sets.

Table 1 Basic Characteristics of Participants

Predictors of MCI Selected in the Derivation Set

In the derivation set, age (61.7 ± 8.4 years vs 55.7 ± 11.6 years) and proportion of female (female/male: 113/172 vs 174/421) were higher and education years were lower (10.5 ± 3.2 years vs 12.8 ± 3.2 years) in patients with MCI than patients cognitively normal (Table 2). In the univariate logistic analysis, fourteen factors were selected as predictors for multivariate analysis (Table 2). These predictors included demographics including age, gender, education and ever smoking, and clinical features including diastolic blood pressure, FPG, HbA_1c, self-reported history of severe hypoglycemia, microvascular disease, peripheral atherosclerosis, hyperlipidemia, cardiovascular disease, cerebrovascular disease, and osteoporosis.

Table 2 Univariate Logistic Regression Analysis of Influence Factors of MCI in T2DM Patients in Derivation Set

The Final Multivariate MCI Prediction Models and MCI Risk Score

A risk model was developed using the selected predictors. The β coefficients, odds ratio and 95% confidence intervals (CI) of the final model are presented in Table 3. Age, education years, self-reported history of severe hypoglycemia, microvascular disease and HbA_1c were independently associated with MCI. In order to facilitate the application of the risk model in clinical practice, it was converted into integer risk scores in which the continuous variables were stratified. Detailed steps have been showed in Supplementary Table 1. Age was categorized in 10 years strata: <40, 40–49, 50–59, 60–69, ≥70 years; education was divided into ≤9 (middle school and below), 10–12 (high school or technical secondary school), ≥13 (college and above) years; for HbA_1c (5.0–7.0%, 7.1–9.0%, ≥9.1%), we used clinically established glycemic control target for categories. The β coefficient of 5 years of age (5×0.048=0.24) was used as a reference standard and assigned one point. Table 4 shows the contribution of each factor in the risk score.

Table 3 Final Multivariate Model of MCI in T2DM Patients

Table 4 Contribution of Predictors in the Diabetes Specific MCI Risk Score

Predicted probability of MCI was reported by levels of sum points of the risk score in Figure 1. The sum points of the risk score ranged from −4 to 20. There was a 18.7-fold difference in MCI probability between the lowest (−4, associated risk 4.3%) and highest (≥19, associated risk 80.3%) sum points.

Figure 1 Predicted probability of MCI by levels of sum points of the risk score in the derivation set. Abbreviation: MCI, mild cognitive impairment.

Performance and Validation of the Risk Score

The AUCs and their 95% CI of the risk score were 0.751 (0.717, 0.784) in the derivation set (Figure 2a) and 0.776 (0.727, 0.824) in the validation set (Figure 2b), respectively. The results demonstrated that the risk score had good discrimination ability. The MCI risk score (AUC: 0.785, 95% CI: 0.730–0.785) was compared to the diabetes-specific dementia risk score (DSDRS)²¹ (AUC: 0.678, 95% CI: 0.648–0.709) in totally 1256 patients with T2DM, and the results are presented in Supplementary Figure 2.

Figure 2 Area under the receiver-operating characteristic curve (AUC) of the MCI risk score. (a) AUC of the risk score in the derivation set, (b) AUC of the risk score in the validation set.

Calibration plots comparing observed and predicted MCI of the risk score showed good agreement both in the derivation (Figure 3a) and validation set (Figure 3b). The results of Hosmer–Lemeshow χ² test for MCI risk in the derivation and validation set all demonstrated excellent goodness-of-fit (P > 0.05). The risk score showed good calibration of observed and predicted MCI probability.

Figure 3 Calibration plots comparing observed and predicted MCI by nine quantile of the risk score. Hosmer-Lemeshow goodness-of-fit test: (a) in the derivation set, χ2= 2.383, P = 0.936; (b) in the validation set, χ2=11.953, P = 0.153. Abbreviation: MCI, mild cognitive impairment.

Youden index of the risk score was 0.380 with a cut-off point of 9, which was corresponding to a 36.6% MCI risk, 69.1% sensitivity and 68.9% specificity in the derivation set, 69.3% sensitivity and 75.5% specificity in the validation set. The sensitivity, specificity, PPV and NPV of the MCI risk score at two cut-off points are presented in Supplementary Table 2. In the derivation set for the risk score, applying a cut-off of ≥4 would identify MCI with high sensitivity of 92.3% and NPV of 91.1%; on the other hand, using a cut-off of ≥14 would identify MCI with high specificity of 94.5% and PPV of 64.9%. Similar results were found in the validation set, suggesting that the risk score was stable

Patients in the derivation set were divided into three groups according to the two cut-off points of the risk score: the low (≤3), medium (4–13), and high (≥14) score group. Applying the low score group as the reference, the risk of MCI significantly increased in both the medium and high score groups (medium score group: OR 6.18, 95% CI: 3.86–9.89; high score group: OR 18.99, 95% CI: 10.33–34.92) (see Supplementary Table 3). Finally, the risk score was summarized in a flowchart to facilitate its application (see Supplementary Figure 3).

Discussion

This study developed a diabetes specific MCI risk score model in adult patients with T2DM at different age. The MCI risk score assesses the risk of MCI in patients with T2DM based on age, education, self-reported history of severe hypoglycemia, microvascular disease and HbA_1c. The risk score for MCI showed good discrimination and calibration ability both in the derivation and validation set. The risk score was capable of stratifying individuals into 3 risk categories by two cut-off points (low risk: ≤3, medium risk: 4–13 and high risk: ≥14) and compared with the low score group, the risk of MCI significantly increased in both the medium and high score group.

One in ten to fifteen cases of dementia is attributable to diabetes, making diabetes as an important target for dementia prevention.²² Several risk factors for cognitive dysfunction in diabetes were determined, including age, lower education level, hypertension, dyslipidemia, depression, diabetes duration, poor glycemic control, and diabetes-related complications, such as the microvascular and macrovascular comorbidities.^23,24 Recently, a study demonstrated that insomnia, suboptimal glycemic control, and non-insulin use were independent risk factors for incident chronic cognitive impairment in older Chinese with T2DM.²⁵ Another Chinese research found major risk factors for T2DM-related cognitive dysfunction were homocysteine, brain infarction, severity of dependence, while education level and creatine kinase were protective factors.²⁶ Previous studies mainly focused on risk factors or biomarkers associated with cognitive dysfunction in patients with diabetes, while multivariate risk prediction models may be valuable tools for detection and prevention of cognitive impairment. Early detection of MCI in patients with diabetes may help to develop and target preventive treatment. However, the implementation of cognitive screening in patients with diabetes is not optimistic. One study showed that 23% of the physicians, 37.4% of the advanced practice nurses, and 8.3% of the physician assistants put routine cognitive screening into their daily practice of diabetes care, and most of them did not use the standardized screening tools with MMSE and MoCA, 57.7% of the participants thought that insufficient time was the most important reason to hinder the screening.¹⁰ The MCI risk score in this study is proposed to be used as a rapid practical prescreening tool of MCI. All the predictors are easily accessible during clinical practice or from the medical records. It does not require specialized tests or expensive and/or labor-intensive procedures, such as cognitive tests, biomarkers or brain imaging. This scoring system will facilitate the implementation of cognitive screening in clinical practice of diabetes care.

A number of risk prediction models for dementia or transition from MCI to dementia have been developed in the general population, while relatively few models for MCI or in patients with diabetes.^27,28 Two dementia prediction models have been established specially for patients with T2DM^21,29 and one for patients with T2DM and hypertension using information of comorbid disease conditions and prescription drugs.³⁰ The DSDRS is the first risk score capable of predicting 10-year individualized dementia risk in elderly patients with T2DM, presenting that a combination of diabetes complications (acute diabetic event, microvascular disease and diabetic foot) and comorbidities (cerebrovascular disease, cardiovascular disease, and depression), education and age were highly predictive of dementia (C statistics 0.733–0.744).²¹ Subsequently, another dementia risk score was developed using clinical data of T2DM patients aged 50–94 years in Taiwan, exhibiting good predictive ability for 3-, 5-, and 10-year dementia risks. It included the following predictors: age, sex, duration of diabetes, BMI, variation in FPG and HbA_1c, stroke, hypoglycemia, postural hypotension, coronary artery disease and anti-diabetes medications (C statistics 0.60–0.76).²⁹ More recently, two cross-sectional studies have developed nomograms to assess the risk of MCI in T2DM patients. One study demonstrated that a nomogram utilizing age, education level and the triglyceride glucose index could predict MCI risk in patients with T2DM (AUC 0.849).¹¹ And the other proposed a nomogram based on age, marital status, per capita monthly household income, diabetes duration, diabetic retinopathy, anxiety, and depression, which showed an optimal diagnosis of MCI (C-index 0.83).¹² Similar to the above models, the risk score in this study was also developed using easily available clinical predictors and presented similar predictive ability. Although both nomogram and risk scoring tables can present the prediction model in an intuitive way, the application of risk scoring tables are more convenient for medical staff in busy clinical practice, such as the well-known Framingham risk score.¹⁹ Notably, the brief scoring system in this study has significant practical utility for assessing MCI risk of T2DM patients and conducting risk stratification.

Predictors in our final model are consistent with previous studies. Like most dementia and MCI risk prediction models,^{11,12,27,28,31} age is strongly associated with an increased risk of MCI, and education is a significant protective factor in our final model. Predictors special for diabetes like HbA1c, history of severe hypoglycemia, and microvascular disease are also included in the final model, which is consistent with views from previous studies showing that the appearance of cognitive dysfunction in T2DM patients is related to chronic hyperglycemia, vascular risk factors and complications, and severe hypoglycemia.³² This also indicates the importance of glycemic control and developing a risk score model special for patients with T2DM.

This is the first time that HbA_1c is included as a predictor in the multivariable risk prediction model for cognitive impairment. HbA_1c is recognized as the gold standard for assessing glycemic control. A cross-sectional study revealed that HbA_1c ≥7.51%, duration of T2DM over 20 years, age above 41 years, and females were independent contributors for cognitive dysfunction in T2DM patients.³³ Prospective studies also have shown that chronic hyperglycaemia based on HbA_1c levels is associated with an increased risk of cognitive decline and incident MCI.³⁴ A recent study found that older Chinese adults with HbA_1c ≥ 7% and insomnia were at 2-fold and 3-fold risk of incident cognitive impairment respectively compared to their counterparts.²⁵ However, results of the Action to Control Cardiovascular Risk in Diabetes (ACCORD) have shown that intensive glycemic control cannot preserve cognitive function or total brain volume after approximately 4 years follow-up.³⁵ That may be related to the increased risk of hypoglycemia with intensive glycemic control. Hypoglycemia has been recognized to be a risk factor for cognitive dysfunction in patients with diabetes.²³ The risk score in this study also included history of severe hypoglycemia as a predictor for MCI, which can supplement valuable information HbA_1c cannot provide.

Unlike the two dementia risk scores for diabetes,^21,29 cardiovascular and cerebrovascular diseases are not included in the risk score. This may be mainly because cognition of the patients was assessed by cognitive tests in this study, some patients with moderate to severe cardiovascular and cerebrovascular diseases usually refused to participant or could not cooperate in completing the cognitive tests. Patients in this study were all from the department of endocrinology, while some patients with severe cardiovascular and cerebrovascular diseases may be hospitalized in other departments. However, the two dementia risk scores identified diagnosis by medical records in retrospective cohort study, so they may include more patients with severe cardiovascular and cerebrovascular diseases or end organ complications.

There are several strengths of our study. First, adult patients with T2DM at different ages were included, so the scoring system has the flexibility to modulate risk points based on age. MCI usually precedes dementia, so earlier detection is required. The application of the MCI risk score is not age-restricted, so it can be used for both middle-aged and elderly patients. Second, all the predictors are easily accessible which is conducive to the application of the scoring system. Third, predictors special for diabetes (HbA_1c and history of severe hypoglycemia) were included, indicating the importance of glycemic control. Notably, the MCI risk score appeared to show a better predictive ability compared to DSDRS in totally 1256 patients with T2DM in this study, which maybe related to the different target population of the models and patients with depression were excluded in our study.

Limitations of the study also must be acknowledged. First, it is a single-centre cross-sectional study. Thus, a multicenter prospective study is needed in the future. Second, though the risk score has been validated in internal validation set, an external validation is needed. The proposed risk score may not be fully generalizable outside our center, so external validation in other population with different backgrounds should be considered in future studies. However, this is an important first step in classifying individuals with T2DM into different categories of MCI risk.

Conclusion

The present study developed a diabetes specific MCI risk score that can be used as a simple, fast, and cost-effective prescreening tool for MCI in daily clinical care of diabetes, to estimate the individual MCI risk and conduct risk stratification. It might be a useful practical tool for early detection and prevention of cognitive dysfunction for adult patients with diabetes and increase the clinicians’ and patients’ awareness of cognitive impairment.