Identifying the Most Crucial Factors Influencing Self-Compassion Among Community-Dwelling Older Adults with Type 2 Diabetes Using Interpretable Machine Learning

Introduction

Population aging has emerged as an irreversible global challenge, and China in particular, experiencing accelerated demographic shift. According to the seventh National Census performed by China’s National Bureau of Statistics, 18.7% of China’s population (260.4 million) is aged 60 or older.¹ This demographic transition has consequently increased the burden of chronic diseases, particularly diabetes, which has become a critical public health priority due to its growing prevalence and serious complications affecting multiple organ systems. The incidence of diabetes in China has increased significantly,² and the number of elderly people with diabetes has reached 35.5 million, representing approximately a quarter of the global elderly diabetic population, ranking first worldwide.³ Beyond its physiological impact, diabetes poses unique challenges for the aging population. These challenges, which extend beyond glycemic control, encompass a higher risk of multimorbidity, polypharmacy, functional decline, cognitive impairment, and increased social isolation.⁴ These age-related and condition-specific factors together increase the risk of psychological distress significantly, making this group especially vulnerable. For instance, studies show that diabetes-related distress affects a large number of people with type 2 diabetes, and older adults may face greater risk due to a higher rate of comorbidities.^4,5 Given this context, it is crucial to recognize that diabetes is a complex condition that encompasses both physiological and psychological dimensions. Despite individuals with diabetes manage their blood glucose levels, they simultaneously confront numerous psychological challenges. As highlighted in a previous research, living with and managing this chronic condition places significant emotional burdens, including, but not limited to, diabetes-specific distress, generalized anxiety, and clinical depression.⁶

The advent of positive psychology has allowed scholars to study how individuals can leverage their inherent positive psychological attributes to mitigate negative emotions. As a crucial component of positive psychology, self-compassion is defined as the capacity to extend kindness and understanding toward oneself, especially during challenging times, including instances of failure, rejection, and loss.⁷ Neff defines self-compassion as consisting of three elements: self-kindness, common humanity, and mindfulness, each of them relevant to the lived experience of having diabetes and coping with its ongoing challenges.^8,9 Individuals with high levels of self-compassion tend to experience greater feelings of happiness, optimism, life satisfaction, and positive affect, compared to low levels of self-compassion that is associated with the presence and severity of mood disorder symptoms.^10–12 A meta-analysis also demonstrated that self-compassion has a significant negative correlation with psychopathological symptoms, effectively alleviating depressive and anxiety symptoms.¹¹ Therefore, self-compassion is a mutable trait that is associated with psychological well-being and the internal drive to participate in beneficial health-related actions.¹³

Previous research highlighted that self-compassion provides a constructive alternative, potentially improving diabetes health outcomes, thus allowing diabetes management.^14,15 Specifically, self-compassion is significantly linked to improved adherence to treatment regimens, reduced hemoglobin A1c (HbA1c) levels, and increased psychological well-being across multiple domains.^16–18 Low levels of self-compassion, especially these aspects of it, influence the decisions of a patient with type 2 diabetes to participate and persist with both self-management and professional-guided care for their condition.¹⁹ The significance of self-compassion is well-documented; however, there is a scarcity of research in China that specifically investigates the key predictors and their relative importance in determining self-compassion levels among community-dwelling older adults with type 2 diabetes. Most existing studies rely on traditional statistical methods, which may not effectively capture complex, non-linear interactions among multiple psychological, social, and clinical factors.^20,21 This gap impedes our capacity to identify the most effective targets for intervention. Furthermore, latest research findings indicate that machine learning possesses robust modeling abilities, a high level of automation, and exceptional accuracy in nonlinear fitting, making it an ideal tool to face complex nonlinear problems.²² These capabilities make it especially suited for investigating the multifaceted nature of self-compassion, which is likely influenced by a web of interrelated variables. Therefore, employing machine learning is not merely a technical choice but a methodological necessity to accurately identify and rank the most crucial factors in this specific population. Machine learning algorithms are better at identifying complex relationships between inputs and outputs when dealing with multiple explanatory variables, resulting in better performance. However, the “black-box” nature of machine learning models often makes it difficult to interpret their internal mechanisms and the logic behind their predictions.²³ To address this limitation and enhance the clinical applicability of our findings, this study employs Shapley Additive Explanations (SHAP), an explainable artificial intelligence method, to interpret the model’s predictions and rank the importance of predictors.²⁴ By using SHAP, we aim to provide healthcare providers with clear, actionable insights into the key factors influencing self-compassion, thereby facilitating the development of precise interventions for community-dwelling older adults with type 2 diabetes and low self-compassion.

This study aimed to identify the most influencing crucial factors of self-compassion among community-dwelling older adults with type 2 diabetes using interpretable machine learning methods. First, six machine learning models were developed and validated to assess the optimal model. Second, based on the best-fitting model, SHAP was used for the global-level analysis of the variables, and the contribution of each variable to the model output was elucidated.²⁵ The ultimate goal of this study was to develop an interpretable prediction model for self-compassion for community-dwelling older adults with type 2 diabetes, providing clinical healthcare providers with a precise risk assessment tool to guide targeted prevention and intervention strategies.

Methods

Study Design

A cross-sectional study was performed among older adults with type 2 diabetes and extended over a month period between July and August 2024. Study participants were recruited from three local communities that belong to a sub-district in Jiaxing, China, selected because they have the highest proportion of elderly people. The recruitment targeted older adults routinely followed up for diabetes management in these communities. Potential participants were identified through community health records and were subsequently contacted by telephone. A total of 320 individuals were initially contacted. They were invited to visit their community health station at a designated time to complete the survey questionnaire and provide a venous blood sample for HbA1c testing. The inclusion criteria included individuals aged 60 years and older who resided in the community and voluntarily agreed to participate in the study. Older adults with any acute complications, severe complications, a history of mental illness, or cognitive or communication disorders were excluded.

This study used convenience sampling, resulting in the collection of 298 valid questionnaires. The final sample of 298 was achieved after 22 individuals were excluded due to personal reasons (eg, time constraints or refusal of blood sampling). Participants were thoroughly informed by staff members about the purpose and procedure of the research before completing the questionnaire. It was also emphasized that all the provided information would remain anonymous and that participants had the right to withdraw from the study at any time.

This study was approved by the Ethics Committee of Affiliated Hospital of Jiaxing University (2024-LY-380) and performed according to the guidelines of the Declaration of Helsinki. All participants provided a written informed consent and voluntarily participated in this study.

Measures

A questionnaire was used to assess self-compassion, depression, and anxiety among community-dwelling older adults with type 2 diabetes.

1. Predesigned structured interview questionnaire was used to collect the following data: socio-demographic information (including age, sex, body mass index, waist circumference, education level, and marital status), personal habits (including experience of negative events and performance of physical exercise), and diabetes history (including duration of diabetes and medication regimen).
2. Self-Compassion Scale-Short Form (SCS-SF)

The 12-item SCS-SF was used to measure self-compassion,²⁶ which consists of 6 dimensions with 2 items each: self-kindness, self-judgment, common humanity, isolation, mindfulness, and over-identification. Participants were instructed to respond to each item using a 5-point Likert scale, with options ranging from 1 (almost never) to 5 (almost always). The participant’s total score ranged from 12 to 60, with higher scores reflecting greater levels of self-compassion. Higher scores for the three positive dimensions indicated greater self-compassion; conversely, higher scores for the three negative dimensions indicated lower levels of self-compassion, characterized by increased self-coldness. The Chinese version of SCS-SF was adapted by Chen.²⁷ In this study, the Cronbach’s alpha of SCS-SF was 0.846.

1. Patient Health Questionnaire (9-item PHQ-9)

Depressive symptoms were measured using the 9-item PHQ-9.²⁸ Participants were instructed to respond to each item using a 4-point Likert scale, with options ranging from 0 (no days) to 3 (almost every day). Participants’ were stratified by the severity of current depressive symptoms according to the PHQ-9 score: none-minimal (0–4 points), mild (5–9 points), moderate (10–14 points), moderately-severe (15–19 points), and severe (20–27 points). The Chinese version of PHQ was adapted by Zheng.²⁹ In this study, the Cronbach’s alpha of SCS-SF was 0.790.

1. Generalized Anxiety Disorder 7-item (GAD-7)

GAD-7 was used to assess anxiety symptoms.³⁰ Participants were instructed to respond to each item using a 4-point Likert scale, with options ranging from 0 (no days) to 3 (almost every day). Participants were stratified by the severity of current depressive symptoms according to the GAD-7 score: mild (5–9 points), moderate (10–14 points), moderately-severe (15–19 points), and severe (20–27 points). The Chinese version of GAD was adapted by Cai.³¹ In this study, the Cronbach’s alpha of GAD-7 was 0.790.

Data Analysis

Machine learning modeling and evaluation were performed using Python (version 3.8). Data preprocessing was performed using the pandas library, while the machine learning algorithm for model training and testing was implemented using scikit-learn (version 1.2.2).

Statistical Analysis

K-means clustering was utilized to define self-compassion subgroups, given the distinct characteristics of the study population (community-dwelling older adults with type 2 diabetes). This data-driven method aimed to ensure that the dichotomization was sensitive to the natural distribution and potential unique manifestations of self-compassion in this specific clinical context. K-means clustering analysis was used to divide self-compassion into a low self-compassion group and a high self-compassion group. The optimal number of clusters was determined using the Elbow Method, where K-Means clustering was applied and the best K value was identified as the one corresponding to the inflection point on the Elbow Plot.

The Shapiro–Wilk test was used for assessing data normality. Normally distributed quantitative data were expressed as mean ± standard deviation, while data not normally distributed were expressed as median and quartiles. Count data were expressed as percentages (%).

Group comparisons between the two self-compassion clusters were conducted to examine differences in demographic and psychological variables. For normally distributed continuous variables, independent samples t-tests were used; for non-normally distributed variables, the Mann–Whitney U-test was applied; and for categorical variables, the chi-square test (or Fisher’s exact test, when appropriate) was used. A p value < 0.05 was considered statistically significant.

Feature Selection and Data Processing

Python random forest algorithm was used to sort variables and build a random forest graph. Variables were added to the model recursively in order of importance to achieve the highest accuracy, and the aforementioned variables were selected as the best feature set. It is noteworthy that the random forest algorithm is inherently robust to potential multicollinearity among features due to its ensemble nature and random feature selection during tree construction.³² Furthermore, the initial feature pool was defined based on clinical relevance prior to the data-driven selection process.

Prediction Model Construction

A computer-generated random number sequence divided the data into training (75%) and validation (25%) cohorts. All models underwent 5-fold cross validation (5-CV) to protect from overfitting. Six supervised classification algorithms were used for self-compassion detection: random forest, support vector machine (SVM), K-Nearest Neighbor (KNN), extreme gradient boosting tree (XGboost), decision tree and naive bayes. The model predictive performance was calculated based on the training results after the cross validation of both testing and training data. Their area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, and Brier were calculated, and compared for self-compassion, class 1 and 2 subgroups, respectively. The comparison was made using ROC analysis, and the results were plotted.

SHAP-Based Interpretations

SHAP was used to interpret the decision-making process of the optimal model, which allowed the extraction of feature analysis and the rational generation of patient-specific prediction.²⁴ This method quantified the contribution of each feature to the individual prediction, enabling both global feature importance analysis and patient-specific outcome explanation. SHAP values operated by measuring how each feature shifted the model’s output from a baseline prediction (the average model output) toward the final result.

Results

K-Means Clustering Analysis for Self-Compassion

According to the K-means clustering analysis, two clusters for self-compassion were found: the low-self-compassion group and the high-self-compassion group (Table 1). The elbow plot revealed that k-means clustering, with the number of clusters set to 2, represented the inflection point of the curve. Hence, K=2 was identified as the elbow and signified its categorization into two distinct groups (Figure 1).

Table 1 K-Means Clustering Analysis for Self-Compassion

Figure 1 Elbow plot.

Demographic Characteristics and Self-Compassion Among Community-Dwelling Older Adults with Type 2 Diabetes

The detailed demographic and clinical characteristics of the participants are summarized in Table 2. Significant differences were observed between the low and high self-compassion groups in several demographic and clinical characteristics (Table 2). Participants with higher self-compassion were more likely to engage in regular physical exercise (p < 0.001) and less likely to report complications (p = 0.005). They also showed higher fasting blood glucose levels compared with the low self-compassion group (median [IQR]: 7.00 [6.50, 7.80] vs 6.70 [6.30, 7.60], p = 0.03). In addition, the high self-compassion group reported significantly lower levels of depressive and anxiety symptoms, as reflected by lower PHQ-9 (p < 0.001) and GAD-7 (p < 0.001) scores.

Table 2 Demographic and Clinical Characteristics of the Participants in the Whole Sample (n = 298) Mean (SD) or n (%)

Model Performance

The Random Forest showed the order of importance of influencing factors for self-compassion (Figure 2). The chart shows the variation in cross-validation accuracy as the number of features increased. The accuracy peaked at seventeen features, suggesting that these initial features had the most significant impact on model performance. Performance was not improved by adding more features beyond this point and could even introduce noise or redundancy (Figure 3). The variable coding and assignment of random forest models are shown in Table 3.

Table 3 Variable Coding and Assignment of Random Forest Models

Figure 2 Influencing factors for self-compassion with random forest in order of importance.

Figure 3 Accuracy variation during feature selection.

The final model, which incorporated these 17 features, was trained on a cohort of 224 participants (75% of the total sample). The performance of each model is summarized in Table 4. The receiver operating characteristic (ROC) curves for the test set are shown in Figure 4. The random forest model performed the best, with an AUC of 0.947, accuracy of 0.872, sensitivity of 0.862, and specificity of 0.809. The Brier score for the random forest was the lowest (0.113), indicating that the model was reliable.

Table 4 Model Performance in Predicting Self-Compassion in the Validation Cohort

Figure 4 ROC curves of different models on the test set.

Given its superior performance, the Random Forest model was selected as the optimal model for this study. It is noteworthy that this model was implemented using scikit-learn’s default hyperparameters without conducting extensive tuning (eg, grid search). This approach was adopted to establish a robust and reproducible baseline while prioritizing generalizability and computational efficiency, which aligned with the primary focus of developing an interpretable prediction tool. The strong performance metrics indicate that the default configuration was effective for the present analysis.

SHAP-Based Interpretations

SHAP analysis was applied to the optimal Random Forest model to interpret its decision-making process. SHAP values were used to uncover the risk factors to elucidate the relationship between the target outcome and positive and negative predictors. The SHAP visualization chart was used to visualize the individual decision-making processes for self-compassion in the optimal random forest model. Figure 5 shows that each row corresponds to a feature and each dot represents a sample; the intensity of the color indicates the magnitude of the feature, with red representing higher values and blue indicating lower values. Depression, HbA1c, waist circumference and anxiety negatively predicted self-compassion; that is, higher depression, HbA1C, waist circumference and anxiety were associated with a lower level of self-compassion. FBG positively predicted self-compassion and higher FBG was linked to a higher level of self-compassion.

Figure 5 SHAP visualization chart of feature variables for the LightGBM.

Discussion

This study described the construction of six machine learning models using selected clinical crucial features to identify the factors that contributed to self-compassion among community-dwelling older adults with type 2 diabetes. The random forest model outperformed the others, with better AUC, accuracy, sensitivity and specificity. It also had the lowest Brier score, suggesting its reliability. Additionally, the random forest model was constructed to show the importance of the variables. Our findings indicated that the depression, HbA1C levels, waist circumference, FBG and anxiety were the top five variables that most significantly influenced the random forest model.

The study identified depression and anxiety as important risk factors for self-compassion among community-dwelling older adults with type 2 diabetes. On the one hand, previous studies showed that the higher the level of self-compassion, the lower the level of both depressive symptoms and diabetes distress.¹³ Our findings broaden the understanding of this relationship to include community-dwelling older adults with type 2 diabetes, a population facing specific challenges, including the management of chronic diseases and age-associated functional decline. This might be related to the following reasons. Beck’s cognitive theory states that the core characteristic of depression lies in negative self-schemas, in which individuals with depression tend to interpret their experiences through a self-critical lens, as if attributing failures to internal shortcomings rather than external factors.³³ Therefore, depressive symptoms may undermine self-compassion through pathways like excessive self-blame or contemplation of health-related losses. This study indicated that self-compassion training (eg, mindfulness-based interventions) could be a new intervention targeting depression among older adults with type 2 diabetes, consistent with Yipa’s demonstration that self-compassion induction (eg, self-care letter writing) significantly enhances rapid recovery from negative stimuli.³⁴ On the other hand, previous research established that daily management of diabetes results in anxiety.³⁵ Anxiety in turn exacerbates emotional dysregulation, often manifesting as hypervigilance toward perceived health threats (eg, an anxiety-prone personality is a predictor for fear of hypoglycemia),^36,37 which may reduce self-compassion by amplifying cognitive rigidity and reducing conscious acceptance of one’s condition. Therefore, integrating psychological assessment into diabetes care for community-dwelling older adults with type 2 diabetes is essential; those with elevated depressive symptoms should receive targeted interventions to increase self-acceptance and preserve self-compassion.

This study discovered that HbA1c was a clinically significant risk factor. Consistent with findings from studies performed in Australia and New Zealand,^16,38 this study demonstrated a significant correlation between increased self-compassion levels and reduced HbA1c concentrations in individuals with diabetes. HbA1c is a commonly used indicator of glycemic management,³⁹ which is frequently internalized by patients as a benchmark to evaluate the effectiveness of self-management. However, the course of self-management consists of intense feelings, such as guilt, frustration, hopelessness, and fear.⁴⁰ Therefore, individuals with diabetes perceive glycemic control failures as personal inadequacies due to the above negative thinking, thus reinforcing self-critical tendencies and thereby destroying individuals’ capacity for self-kindness and emotional acceptance in diabetes management. Critically, these self-critical tendencies are not merely psychological; they can translate into tangible behavioral outcomes, such as avoidance of glucose monitoring or poor treatment adherence, which directly contribute to suboptimal glycemic control.^41,42 Consequently, healthcare providers should integrate targeted diabetes education programs focused on promoting non-judgmental attitudes toward glycemic fluctuations and reducing self-blame, thereby preserving patients’ ability of self-kindness and adaptive emotional acceptance in diabetes management.

Importantly, this study made an intriguing and counterintuitive finding: higher FBG levels were associated with increased levels of self-compassion. This suggests that in this specific population, FBG may play a more complex role than merely a biomarker of poor control, potentially acting as a unique protective factor. One plausible explanation lies in self-efficacy theory. When community-dwelling older adults with type 2 diabetes achieve FBG levels within their perceived acceptable range, it creates a powerful mastery experience. This immediate and observable feedback on recent self-management behaviors in turn serves to enhance their confidence in managing the condition, known as self-efficacy, as outlined by Bandura’s theory.⁴³ Such enhanced self-efficacy could mitigate self-criticism and create emotional space for self-compassion.⁴⁴ However, this positive association warrants careful interpretation. It may also reflect more complex psychological adaptations, as explained by response shift theory. According to this framework, patients with persistently elevated glucose levels may recalibrate their internal standards of “success” and “health,” thereby preserving well-being despite suboptimal control.^45–47 From a self-regulatory standpoint, this represents a form of cognitive restructuring to maintain emotional equilibrium.⁴⁸ Clinically, these insights urge healthcare providers to acknowledge the varied meanings patients assign to glucose values and to adopt individualized approaches to interpreting both metabolic and psychological data.

Waist circumference was also identified as an important risk factor for self-compassion among community-dwelling older adults with type 2 diabetes. Waist circumference is a reliable indicator for assessing central obesity in diabetic patients and is significantly associated with metabolic risk. Therefore, monitoring waist circumference is necessary for tracking treatment utility and intervention effectiveness, providing insights beyond BMI alone.⁴⁹ Obese people are prone to social stigmatization and internalize negative evaluations, leading to self-criticism. The profound impact of weight stigma lies in its erosion of an individual’s self-concept. According to self-discrepancy theory, a significant gap between one’s actual body image (as indicated by waist circumference) and societally promoted “ideal” body standards can trigger intense shame and self-criticism, directly undermining the core components of self-kindness and self-acceptance.⁵⁰ A systematic review indicated that the main consequence of weight stigma is reduced self-esteem in overweight and obese adults, compounded by increased eating disorders, depressive symptoms, anxiety, and body dissatisfaction.⁵¹ Moreover, many individuals who experience weight stigma internalize these experiences, directing the stigma inward and placing the blame on themselves.⁵² This vulnerability to stigma and its internalization is particularly acute in individuals with type 2 diabetes, who report higher rates of both weight stigma and diabetes-related stigma than the general population.⁵³ This process has been shown to undermine the cultivation of self-compassion through the internalization of stigmatizing statements. Therefore, healthcare providers are recommended to incorporate psychosocial interventions designed to mitigate the internalization of weight stigma and foster body acceptance, thereby safeguarding the capacity for self-compassion among diabetic patients throughout the disease management process.

In summary, the strength of the machine learning approach employed in this study lies in its ability to move beyond the isolated analysis of individual factors and reveal the intertwined risk dynamics among variables. The random forest model does more than rank variable importance; it inherently captures complex, non-linear interactions between features through its branching logic and ensemble structure. Its predictions stem from decisions made across numerous subsets of features and data, and the aggregation of these predictions implicitly models the joint effects of the features.^54,55 In this study, the interactions and potential synergies observed among depression, anxiety, HbA1c, fasting blood glucose, and waist circumference indicate that the level of self-compassion in older adults with diabetes is not eroded by any single factor in isolation, but rather by the combined impact of psychological distress, metabolic dysregulation, and body image concerns. This systems-level insight, which would be difficult to pre-specify and capture using traditional logistic regression, strongly argues for an integrated clinical intervention model: a collaborative team consisting of endocrinologists, mental health professionals, and dietitians should be established to develop unified treatment plans for patients, ensuring that glycemic management and psychological interventions mutually reinforce each other, rather than operating in separate silos.

Strengths and Limitations

The main strength of our study was the use of an interpretable machine learning approach. This study developed a predictive framework through interpretable machine learning, combining random forest modeling with SHAP interpretability techniques. The integrated methodology effectively resolved complex inter-variable interactions while maintaining robust prediction stability. Critical predictors were identified using the random forest algorithm, with SHAP analysis precisely quantifying the impact of each factor on self-compassion risk in community-based elderly type 2 diabetes patients. This transparent analytical approach provided clinically actionable insights, enabling healthcare practitioners to design targeted interventions to improve self-compassion in this population.

Despite the encouraging results, this study has several limitations. First, the cross-sectional design limited our ability to dynamically track self-compassion trajectories or establish causal relationships. Furthermore, this design limits our ability to elucidate the underlying mechanisms behind the counterintuitive association between FBG and self-compassion. Future longitudinal studies are needed to verify this finding and clarify its cognitive-affective pathways. Second, the current sampling was confined to several specific communities in Jiaxing, compromising the generalizability and robustness of the findings. Third, the model was developed with a limited sample size relative to the number of predictors, which may affect the stability of the identified features. External validation in larger, multi-center cohorts is essential to confirm the model’s robustness and clinical applicability.

Conclusion

This study constructed a risk prediction model of self-compassion among community-dwelling older adults with type 2 diabetes using interpretable machine learning methods. The SHAP analysis based on the random forest model identified depression, HbA1C, waist circumference, and anxiety as risk factors for self-compassion and FBG as a protective factor; consequently, targeted interventions were proposed. This study provides healthcare providers with a data-driven screening tool to identify factors influencing psychological adjustment in older adults, offering evidence-based support for the development of targeted interventions to improve the level of self-compassion. Moving forward, these findings could be translated into a practical risk scorecard based on the most influential SHAP features, facilitating efficient patient screening and personalized interventions in community health centers.