Back to Journals » Neuropsychiatric Disease and Treatment » Volume 16

Identifying Suicidal Ideation Among Chinese Patients with Major Depressive Disorder: Evidence from a Real-World Hospital-Based Study in China

Authors Ge F , Jiang J, Wang Y, Yuan C, Zhang W

Received 12 November 2019

Accepted for publication 21 January 2020

Published 4 March 2020 Volume 2020:16 Pages 665—672


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 3

Editor who approved publication: Dr Yuping Ning

Fenfen Ge,1,* Jingwen Jiang,2,* Yue Wang,1 Cui Yuan,1 Wei Zhang1

1Mental Health Center of West China Hospital, Sichuan University, Chengdu, Sichuan 610041, People’s Republic of China; 2West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, People’s Republic of China

*These authors contributed equally to this work

Correspondence: Wei Zhang
Mental Health Center of West China Hospital, Sichuan University, Chengdu, Sichuan 610041, People’s Republic of China
Tel +86 18980601010
Email [email protected]

Background: A growing body of research suggests that major depressive disorder (MDD) is one of the most common psychiatric conditions associated with suicide ideation (SI). However, how a combination of easily accessible variables built a utility clinically model to estimate the probability of an individual patient with SI via machine learning is limited.
Methods: We used the electronic medical record database from a hospital located in western China. A total of 1916 Chinese patients with MDD were included. Easily accessible data (demographic, clinical, and biological variables) were collected at admission (on the first day of admission) and were used to distinguish SI with MDD from non-SI using a machine learning algorithm (neural network).
Results: The neural network algorithm distinguished 1356 out of 1916 patients translating into 70.08% accuracy (70.68% sensitivity and 67.09% specificity) and an area under the curve (AUC) of 0.76. The most relevant predictor variables in identifying SI from non-SI included free thyroxine (FT4), the total scores of Hamilton Depression Scale (HAMD), vocational status, and free triiodothyronine (FT3).
Conclusion: Risk for SI among patients with MDD can be identified at an individual subject level by integrating demographic, clinical, and biological variables as possible as early during hospitalization (at admission).

Keywords: depression, suicide ideation, real-world, machine learning


Globally, suicide is a leading public health problem, accounting for 1.4% of the global mortality in 2008.1 In China, national survey shows suicide was the fifth leading cause of death,2 and the prevention of suicide is an important public health strategy.3 Suicidal behavior, which including suicidal ideation (SI), suicide plan, suicide attempt, and completed suicide (CS).4,5 CS is a major risk among patients with major depressive disorder (MDD), where 3.8% of females and 6.7% of males die via suicide.6 Although the fact that CS is a preventable outcome, this phenomenon is limited understood because it is difficult to attain adequate information on deaths by suicide. Considering SI may be a major predictor of the CS among MDD patients7 and suicide completers with MDD usually had SI,8 thus much of the research associated with suicidal behavior pays attention to SI among MDD.

Risk factors for SI specific to MDD include vocational/employment and marital status,9,10 gender,1114 age at onset,15 depression severity,9,11,15,16 comorbidity with other mental disorder17 and lower hypothalamic-pituitary-thyroid (HPT) axis activity.18 Several studies have explored the prevalence of SI in patients with MDD, the prevalence ranged from 18.0% to 58.0%.13,19 A recent meta-analysis indicates that the pooled lifetime prevalence of SI was 53.1% and one-month prevalence of SI was 27.5% among MDD in China.20

We found two limits in the previous studies of identifying risk factors for SI among MDD patients. First, most of the data analysis SI, risk factors are discovered using either univariate association or multivariate analyses methods. The above methods, do not guarantee predictive optimality and do not guarantee parsimony in a data analysis-independent manner.21 The best way to effectively integrate these variables is an open question. Machine learning can be employed for the prediction of later clinical outcomes via combining multiple pieces of information from different domains in an effective way and allowing the identification of the most predictive factors. Secondly, most studies explore SI through strict inclusion and exclusion criteria. Limited studies have explored SI based on real-world data.

In sum, the main purpose of the current research was to establish a clinically useful predictive model to determine individual MDD patients’ likely to have a SI as possible as early (at admission). Easily accessible data (demographic, clinical, and biological variables) were collected at admission and used to achieve our objective.


Data Source and Study Population

The Ethics Committee of the West China Hospital, Sichuan University approved the research. Institute review board (IRB) did not require informed consent from patients. Because this is a retrospective study and we did not use any identifying information of the patient. We keep patient information confidential. And the treatment complies with the Helsinki Declaration. The protection and treatment of patient data in my research comply with the Helsinki Declaration. We used the Medical Record System from West China Hospital of Sichuan University. The database includes claims for inpatient and outpatient diagnoses. In this study, depression was defined as a recorded discharge diagnosis according to the International Classification of Disease, Tenth Revision (ICD-10, Clinical Modification Codes F32 and F33). And we only included people who completed the evaluation of Hamilton Depressive Rating Scale (HAMD), Hamilton Anxiety Rating Scale (HAMA), and the HPT-axis at admission. Patients with a recorded diagnosis of another mental disorder and those with physical diseases were excluded. Figure 1 shows the process of data extraction.

Figure 1 The process of data extraction.


In this study 17-item HAMD and 14-item HAMA were employed. HAMD item 3 was used to assess SI. This item has the following alternative statements: 0=absent, 1=feels life is not worth living, 2=any thoughts of possible death of herself/himself, 3=suicide ideation/suicide gesture, and 4=attempts at suicide. In this research, we defined someone had SI via a score of ≥3 on HAMD item.22,23 In other words, if a patient had scores of 3 or 4 on HAMD item3 he/she would classify into a group with SI, patients with lower scores being allocated into group without SI.

Statistical Analysis

A machine learning algorithm (neural network) and descriptive data analysis were implemented in R 3.60 for Windows. A machine learning approach (neural network) was used to identify the most predictive of SI. Neural network is used for supervised learning problems where we use the training data (with multiple features) Xi to predict a target variable Yi. Specifically, we split data into training (75%) and test (25%) datasets and preserved the proportion of the categories in Y variable. The training dataset was used to build a neural network model, and randomly performed the model with 10-fold cross-training, repeated 3 times and validation to find out the optimal parameters. Meanwhile, since the number of categories is imbalanced (with IS/without SI=319/1597), the downsampling method was also used in the training model. The final-trained model is then used to predict the Y in the testing dataset by looking only X values of the testing dataset.


Prevalence of SI in the Patients

A total of 1916 MDD patients, comprising 1325 (69.2%) female and 591 (30.8%) male patients, ranging from 18 to 70 years old (45.2±15.4). The prevalence of SI was 16.7% (319/1916) in patients with MDD. The basic characteristics at admission are shown in Table 1.

Table 1 Basic Characteristics Between the Patients with SI and Without SI

Distinguishing SI from Non-SI

The machine learning of neutral network identified 1354 out of 1916 as either SI or non-SI and this translated into 70.08% accuracy (70.68% sensitivity and 67.09% specificity). The neutral network receiver operating characteristic (ROC) curve and the “confusion matrix”, which were used in calculating the sensitivity (70.68%), specificity (67.09%), and area under ROC curve values (0.74), are shown in Figures 2 and 3, respectively. The most relevant predictor variables in determining SI from non-SI that were assigned positive weighting or coefficients factors via neutral network suggesting an increase in SI included: 1) FT4, 2) HAMD_total, 3) vocational status (professional skill worker), 4) FT3. Figure 4 demonstrates predictor variables with corresponding weighting factors.

Figure 2 Neural network performance in identifying individual SI from non-SI among MDD.

Figure 3 The AUCs for distinguishing SI from non-SI.

Figure 4 Bar graph showing “feature importance” in distinguishing SI from non-SI.


Prevalence Rate of SI Among MDD

The prevalence rate of SI among MDD was 16.7%, which was significantly higher than in the Chinese general population (3.1%).24 This result indicating that the prevalence of SI in Chinese patients with MDD is high. However, the reported prevalence rate of SI among MDD patients is highly inconsistent, ranging from 11% to 63%.17,25 The substantial variability of prevalence rates could be accounted for assessment methods applied (e.g. HAMD item3, Columbia Classification Algorithm of Suicide Assessment and the Scale for Suicide Ideation)17,26 and the level of the treatment setting (e.g. patients in psychiatric settings and in primary care).27

Risk Factors of SI Among MDD

Dysregulation of HPT axis has been done to research the neurobiological association of suicidal behaviour. Our results suggest that serum FT4 and FT3 levels among SI with MDD were significantly lower than those among non-SI, and these results are consistent with those of previous studies.18,28 Why FT3 and FT4 are related to SI? The dysregulation of serotonergic has been widely studied in adults and has been postulated as a biological marker for suicide.2932 Serotonin regulates growth and maturation of some cerebral regions (e.g. hippocampal neurogenesis) in the developing brain,3335 and it influences secretion of TSH and thyrotropin-releasing hormone (TRH). Jokinen’s study found that depressive subjects with SI or suicide attempt showed a declined TSH response to morning administration of TRH.36 Thus, a deficit in central TRH function in suicide attempters could result in an impaired TSH resynthesise, explaining why FT418 and FT3 are decreased.37,38

Previous studies have explored the relationship at group level between depressive symptom severity and SI among MDD. Consistent with previous studies, the severity of depressive symptoms has a robust association with SI.23 Lim found that patients who had high MADRS score (Montgomery-Asberg Depression Rating Scale, MADRS) had increased odds (adjusted OR=1.11, 95% CI: 1.07–1.15) of being in the high suicidality group.9 A recent meta-analysis of 166 studies published from 1971 to 2014 found that depressive symptoms (OR=2.99, 95% CI: 1.97–4.55) were major predictors of SI.39

In the previous epidemiological and clinical researches, vocational status in SI among MDD have been extensively revealed at group level. And there is accumulating evidence that risk of suicide is increased in medically related professions. Studies in different countries have suggested that suicide risk is elevated in doctors.4043 But not all studies have found that suicide risk is elevated in nurse.4346 Why there is not a risk of suicide in doctors and nurses in China? Almost 70% doctors work more than 50 hrs per week in hospital in China47 and working long hours are more prone to have a depression. The results of the survey of the Chinese Medical Doctor Association (CMDA) suggested that 64.9% doctors showed sign of depression in Shanghai, China.48 However, Chinese are always conservative, and Chinese doctors and nurses are more often ashamed to treat their own mental illness.49,50 In general, medical-related professions think that seeking treatment of mental illness for themselves may subject to stigma, shame, or worse.51 Thus, we did not find medical-related professions as a risk factor for suicidal ideation.

Clinical Implications

We have identified utility clinical models. And our models offer information about the most important variables (FT3, FT4, the severity of depressive symptom and work status) to distinguish SI at admission. Thus, enabling a more efficient assessment process of SI among MDD as possible as early during hospitalization. These results suggest that psychiatrists should pay attention to certain types of MDD (low FT3, low FT4, depressive symptoms are more serious and engaged in medical-related work) patients to reduce the likelihood of suicide behaviour in such patients.

Strengths and Limitations

The current research has some potential strengths, including 1) integrating information from demographic, clinical, and biological variables, 2) focusing on distinguishing SI among MDD patients at admission, and 3) testing models for individual-level classification via machine learning. However, there are some limitations in the study: 1) our patients from a specific western hospital might limit the generalizability of our results, and 2) the data of SI were collected through use of HAMD item 3 rather than structured specific SI instrument. Thus, the severity of SI assessed is not sufficiently systematic and reliable. Therefore, replications based on structured specific SI instrument are also needed. 3) the duration of untreated depression may have a fundamental impact on the clinical outcomes of depressed patients with a shorter duration of untreated depression which is linked with unfavourable outcomes in depressed individuals, including depression-related disability (e.g. suicidal ideation, CS). Future studies should consider this variable when exploring the SI with MDD.52 4) we did not distinguish between those with single and recurrent illness episodes and future studies can clarify this problem.


In conclusion, we suggested that SI among MDD patients can be predicted using machine learning approach at the time of admission. And several of the most relevant variables (e.g. FT4, HAMD_total, vocational status, and FT3) revealed in the current study.


The authors report no conflicts of interest in this work.


1. Värnik P. Suicide in the world. Int J Environ Res Public Health. 2012;9(3). doi:10.3390/ijerph9020343

2. Phillips MR, Yang G, Zhang Y, Wang L, Ji H, Zhou M. Risk factors for suicide in China: a national case-control psychological autopsy study. Lancet. 2002;360(9347):1728–1736. doi:10.1016/S0140-6736(02)11681-3

3. Power AK, McKeon R. Preventing suicide is a national imperative. Am J Public Health. 2012;102(Suppl 1):S7. doi:10.2105/AJPH.2012.300648

4. Scocco P, de Girolamo G, Vilagut G, Alonso J. Prevalence of suicide ideation, plans, and attempts and related risk factors in Italy:: results from the European Study on the Epidemiology of Mental Disorders-World Mental Health study. Compr Psychiatry. 2008;49(1):13–21. doi:10.1016/j.comppsych.2007.08.004

5. Yoshimasu K, Kiyohara C, Miyashita K, The Stress Research Group of the Japanese Society for H. Suicidal risk factors and completed suicide: meta-analyses based on psychological autopsy studies. Environ Health Prev Med. 2008;13(5):243–256. doi:10.1007/s12199-008-0037-x

6. Nordentoft M, Mortensen PB, Pedersen CB. Absolute risk of suicide after first hospital contact in mental disorder. JAMA Psychiatry. 2011;68(10):1058–1064.

7. Park E-H, Hong N, Jon D-I, Hong HJ, Jung MH. Past suicidal ideation as an independent risk factor for suicide behaviours in patients with depression. Int J Psychiatry Clin Pract. 2017;21(1):24–28. doi:10.1080/13651501.2016.1249489

8. Fang X, Zhang C, Wu Z, et al. Prevalence, risk factors and clinical characteristics of suicidal ideation in Chinese patients with depression. J Affect Disord. 2018;235:135–141. doi:10.1016/j.jad.2018.04.027

9. Lim AY, Lee AR, Hatim A, et al. Clinical and sociodemographic correlates of suicidality in patients with major depressive disorder from six Asian countries. BMC Psychiatry. 2014;14:37. doi:10.1186/1471-244X-14-37

10. Brown GK, Beck AT, Steer RA, Grisham JR. Risk factors for suicide in psychiatric outpatients: a 20-year prospective study. J Consult Clin Psychol. 2000;68(3):371–377. doi:10.1037/0022-006X.68.3.371

11. Hawton K, Casañas I Comabella C, Haw C, Saunders K. Risk factors for suicide in individuals with depression: a systematic review. J Affect Disord. 2013;147(1):17–28. doi:10.1016/j.jad.2013.01.004

12. Kwon A, Song J, Yook KH, et al. Predictors of suicide attempts in clinically depressed Korean adolescents. Clin Psychopharmacol Neurosci. 2016;14(4):383–387. doi:10.9758/cpn.2016.14.4.383

13. Sokero TP, Melartin TK, Rytsala HJ, et al. Suicidal ideation and attempts among psychiatric patients with major depressive disorder. J Clin Psychiatry. 2003;64(9):1094–1100. doi:10.4088/JCP.v64n0916

14. Schaffer A, Levitt AJ, Bagby RM, Kennedy SH, Levitan RD, Joffe RT. Suicidal ideation in major depression: sex differences and impact of comorbid anxiety. Can J Psychiatry. 2000;45(9):822–826. doi:10.1177/070674370004500906

15. Zhu Y, Zhang H, Shi S, et al. Suicidal risk factors of recurrent major depression in han Chinese women. PLoS One. 2013;8(11):e80030. doi:10.1371/journal.pone.0080030

16. Holma KM, Melartin TK, Haukka J, Holma IA, Sokero TP, Isometsa ET. Incidence and predictors of suicide attempts in DSM-IV major depressive disorder: a five-year prospective study. Am J Psychiatry. 2010;167(7):801–808. doi:10.1176/appi.ajp.2010.09050627

17. Morris DW, Trivedi MH, Husain MM, et al. Indicators of pretreatment suicidal ideation in adults with major depressive disorder. Acta Psychiatr Scand. 2010;121(6):480–484. doi:10.1111/j.1600-0447.2009.01516.x

18. Peng R, Dai W, Li Y. Low serum free thyroxine level is correlated with lipid profile in depressive patients with suicide attempt. Psychiatry Res. 2018;266:111–115. doi:10.1016/j.psychres.2018.05.059

19. Zhou XD, Li L, Yan Z, Hesketh T. High sex ratio as a correlate of depression in Chinese men. J Affect Disord. 2013;144(1–2):79–86. doi:10.1016/j.jad.2012.06.009

20. Dong M, Wang SB, Li Y, et al. Prevalence of suicidal behaviors in patients with major depressive disorder in China: a comprehensive meta-analysis. J Affect Disord. 2018;225:32–39. doi:10.1016/j.jad.2017.07.043

21. Salcioglu E, Basoglu M. Psychological effects of earthquakes in children: prospects for brief behavioral treatment. World J Pediatr. 2008;4(3):165–172. doi:10.1007/s12519-008-0032-8

22. Vuorilehto M, Valtonen HM, Melartin T, Sokero P, Suominen K, Isometsä ET. Method of assessment determines prevalence of suicidal ideation among patients with depression. Eur Psychiatry. 2014;29(6):338–344. doi:10.1016/j.eurpsy.2013.08.005

23. Li X, Liu H, Hou R, et al. Prevalence, clinical correlates and IQ of suicidal ideation in drug naive Chinese Han patients with major depressive disorder. J Affect Disord. 2019;248:59–64. doi:10.1016/j.jad.2018.12.017

24. Lee S, Fung SC, Tsang A, et al. Lifetime prevalence of suicide ideation, plan, and attempt in metropolitan China. Acta Psychiatr Scand. 2007;116(6):429–437. doi:10.1111/acp.2007.116.issue-6

25. Zisook S, Lesser IM, Lebowitz B, et al. Effect of antidepressant medication treatment on suicidal ideation and behavior in a randomized trial: an exploratory report from the Combining Medications to Enhance Depression Outcomes Study. J Clin Psychiatry. 2011;72(10):1322–1332. doi:10.4088/JCP.10m06724

26. Valtonen HM, Suominen K, Sokero P, et al. How suicidal bipolar patients are depends on how suicidal ideation is defined. J Affect Disord. 2009;118(1):48–54. doi:10.1016/j.jad.2009.02.008

27. Vuorilehto MS, Melartin TK, Isometsä ET. Suicidal behaviour among primary-care patients with depressive disorders. Psychol Med. 2005;36(2):203–210. doi:10.1017/S0033291705006550

28. Asadikaram G, Khaleghi E, Sayadi A, et al. Assessment of hormonal alterations in major depressive disorder: a clinical study. PsyCh J. 2019. doi:10.1002/pchj.290

29. Makris GD, Reutfors J, Larsson R, et al. Serotonergic medication enhances the association between suicide and sunshine. J Affect Disord. 2016;189:276–281. doi:10.1016/j.jad.2015.09.056

30. Park YM. Relationship between serotonergic dysfunction based on loudness dependence of auditory-evoked potentials and suicide in patients with major depressive disorder. Psychiatry Investig. 2015;12(4):421–424. doi:10.4306/pi.2015.12.4.421

31. Sudol K, Mann JJ. Biomarkers of suicide attempt behavior: towards a biological model of risk. Curr Psychiatry Rep. 2017;19(6):31. doi:10.1007/s11920-017-0781-y

32. Picouto MD, Villar F, Braquehais MD. The role of serotonin in adolescent suicide: theoretical, methodological, and clinical concerns. Int J Adolesc Med Health. 2015;27(2):129–133. doi:10.1515/ijamh-2015-5003

33. Whitaker-azmitia PM. Serotonin and brain development: role in human developmental diseases. Brain Res Bull. 2001;56(5):479–485. doi:10.1016/S0361-9230(01)00615-3

34. Jacobs BL, van Praag H, Gage FH. Adult brain neurogenesis and psychiatry: a novel theory of depression. Mol Psychiatry. 2000;5(3):262–269. doi:10.1038/

35. Hasler G. Pathophysiology of depression: do we have any solid evidence of interest to clinicians? World Psychiatry. 2010;9(3):155–161. doi:10.1002/wps.2010.9.issue-3

36. Jokinen J, Samuelsson M, Nordström A-L, Nordström P. HPT axis, CSF monoamine metabolites, suicide intent and depression severity in male suicide attempters. J Affect Disord. 2008;111(1):119–124. doi:10.1016/j.jad.2008.02.001

37. Duval F, Mokrani MC, Erb A, Gonzalez Opera F, Calleja C, Paris V. Relationship between chronobiological thyrotropin and prolactin responses to protirelin (TRH) and suicidal behavior in depressed patients. Psychoneuroendocrinology. 2017;85:100–109. doi:10.1016/j.psyneuen.2017.07.488

38. Mann JJ, Currier D. A review of prospective studies of biologic predictors of suicidal behavior in mood disorders. Archiv Suicide Res. 2007;11(1):3–16. doi:10.1080/13811110600993124

39. Ribeiro JD, Huang X, Fox KR, Franklin JC. Depression and hopelessness as risk factors for suicide ideation, attempts and death: meta-analysis of longitudinal studies. Br J Psychiatry. 2018;212(5):279–286. doi:10.1192/bjp.2018.27

40. Agerbo E, Gunnell D, Bonde JP, Mortensen PB, Nordentoft M. Suicide and occupation: the impact of socio-economic, demographic and psychiatric differences. Psychol Med. 2007;37(8):1131–1140. doi:10.1017/S0033291707000487

41. Hem E. Suicide among doctors. Tidsskr nor Laegeforen. 2015;135(4):305. doi:10.4045/tidsskr.15.0176

42. Schernhammer ES, Colditz GA. Suicide rates among physicians: a quantitative and gender assessment (meta-analysis). Am J Psychiatry. 2004;161(12):2295–2302. doi:10.1176/appi.ajp.161.12.2295

43. Meltzer H, Griffiths C, Brock A, Rooney C, Jenkins R. Patterns of suicide by occupation in England and Wales: 2001-2005. Br J Psychiatry. 2008;193(1):73–76. doi:10.1192/bjp.bp.107.040550

44. Hawton K, Simkin S, Rue J, et al. Suicide in female nurses in England and Wales. Psychol Med. 2002;32(2):239–250. doi:10.1017/S0033291701005165

45. Andersen K, Hawgood J, Klieve H, Kolves K, De Leo D. Suicide in selected occupations in Queensland: evidence from the State suicide register. Aust N Z J Psychiatry. 2010;44(3):243–249. doi:10.3109/00048670903487142

46. Hawton K, Agerbo E, Simkin S, Platt B, Mellanby RJ. Risk of suicide in medical and related occupational groups: a national study based on Danish case population-based registers. J Affect Disord. 2011;134(1–3):320–326. doi:10.1016/j.jad.2011.05.044

47. Wang Y, Liu L, Xu H. Alarm bells ring: suicide among Chinese physicians: a STROBE compliant study. Medicine (Baltimore). 2017;96(32):e7790. doi:10.1097/MD.0000000000007790

48. Association CMD. World suicide prevention day: how can health care workers be so fragile? 2014. Available from: Accessed February 02, 2020.

49. Lindeman S, Laara E, Vuori E, Lonnqvist J. Suicides among physicians, engineers and teachers: the prevalence of reported depression, admissions to hospital and contributory causes of death. Acta Psychiatr Scand. 1997;96(1):68–71. doi:10.1111/acp.1997.96.issue-1

50. Hawton K, Malmberg A, Simkin S. Suicide in doctors. A psychological autopsy study. J Psychosom Res. 2004;57(1):1–4. doi:10.1016/S0022-3999(03)00372-6

51. Wang Y, Liu L, Xu HB. Alarm bells ring: suicide among Chinese physicians A STROBE compliant study. Medicine. 2017;96(32):3.

52. Ghio L, Gotelli S, Cervetti A, et al. Duration of untreated depression influences clinical outcomes and disability. J Affect Disord. 2015;175:224–228. doi:10.1016/j.jad.2015.01.014

Creative Commons License This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.