The Utility of ChatGPT in Diabetic Retinopathy Risk Assessment: A Comparative Study with Clinical Diagnosis [Letter]

Dear editor

I have read a research article entitled “The Utility of ChatGPT in Diabetic Retinopathy Risk Assessment: A Comparative Study with Clinical Diagnosis” by Raghu et al,¹ recently published in Clinical Ophthalmology Journal. I congratulate the authors on this successful article and make some contributions. There are five strengths of this study: 1) This study used ChatGPT-4 to generate a medical report suggesting DR risk based on clinical and biochemical parameters, 2) This study used anonymized data from 111 diabetic patients to evaluate the prediction accuracy of ChatGPT-4 against clinical diagnosis, while also examining the response reliability of ChatGPT-4, 3) This study demonstrated that ChatGPT has the ability to predict DR risk based on clinical and biochemical parameters, with moderate sensitivity and specificity, 4) This study highlights the potential application of ChatGPT as a screening tool with a sensitivity of 73% and specificity of 54% without CST information, and a sensitivity of 67% and specificity of 68% when CST information is included, 5) this study suggests that ChatGPT can be an auxiliary tool in disease detection and monitoring, as well as highlighting the need for further research to improve the accuracy of ChatGPT in DR risk assessment.

However, we have also discovered several limitations that need to be corrected in the future, namely: 1) The ChatGPT language model used in this study is a research version that has not been specifically designed for medical applications, so it may produce nonsensical or incorrect information due to limitations in training data and algorithms, 2) ChatGPT has not been fully optimized in determining the presence or absence of DR, showing only fair agreement between ChatGPT prediction and clinical diagnosis, 3) ChatGPT has moderate sensitivity and specificity, suggesting that there is still room for improvement in producing more accurate results, 4) ChatGPT is not yet fully compliant with HIPAA regulations and is vulnerable to hacking attempts, which raises concerns regarding patient privacy and data protection.

To obtain better results, we recommend that further research be carried out by 1) Further research could consider developing a ChatGPT model that is more optimized for medical applications, possibly through collaboration with healthcare professionals, 2) Further research should be conducted with a larger sample size to increase the statistical power and validity of the study’s conclusions, 3) Future research could consider improvements in complying with data privacy regulations, such as HIPAA,² and strengthening patient data security in the use of ChatGPT, 4) Further research is needed to explore other applications of AI models in healthcare, as well as ways to improve the accuracy of those models, 5) It is important to consider the ethical implications of the use of AI in healthcare, including data privacy and the potential replacement of human decision-making in healthcare.^3,4

In conclusion, this study makes a significant contribution by demonstrating the potential of ChatGPT in predicting the risk of diabetic retinopathy based on clinical and biochemical parameters, although it still requires further research to improve its accuracy and applicability in clinical practice.