Evaluation of an Acne Severity Grading Self-Assessment System Suitable for the Thai Population – A Pilot Study

Introduction

Acne vulgaris (acne) is a common dermatological disorder in young children that frequently persists throughout adulthood. It is estimated that acne affects 9.4% of the world’s population, making its prevalence the eighth highest worldwide.¹ The pathogenesis of acne is multifactorial including multiple causes, both intrinsic and extrinsic. The primary causes consist of follicular hyperkeratosis and excessive sebum production. Genetic factors also play a significant role, with approximately 50% of adult acne patients having confirmed cases of acne in their first-degree family. This genetic connection in acne development is further supported by cytogenetic studies. Additionally, Cutibacterium acnes, superinfection with co-infections with Staphylococcus aureus, and the presence of lipophilic yeasts (Malassezia furfur) also play role in the occurrence of acne.² The prevalence of acne varies in different countries and ethnicities.³ In Thailand, acne is one of the most prevalent diseases, affecting 65% of adolescents.⁴ This condition has been significantly associated with negative psychosocial consequences, such as poor self-image, depression, and anxiety.⁵ Standardization of an acne severity grading system is important in clinical practice because the severity of acne is the most essential factor that influences treatment selection and assessment of treatment response. Grading of acne is also important for clinical, epidemiology and research studies that assess the general population. Generally, an ideal acne grading system should be simple, accurate and allows a quick assessment. Currently, there are many acne grading systems that broadly classify acne based on lesion counting and global assessment however there is no universally accepted acne grading system. Lesion counting is an impractical method because it is tedious, time-consuming and dependent on many variables, such as evaluators’ visual capacity. Global assessment is less time-consuming and recommended in systematic reviews to establish a consistency in clinical practice, clinical trials, and epidemiological surveys because of its simplicity, reproducibility, and good reliability for assessing acne. Additionally, it can distinguish the various clinical manifestations of acne, including the concentration, distribution, size, and signs of inflammation, which cannot be detected by lesion counting.^6–8 Thus, we believe that the global assessment is superior to lesion counting in terms of the clinical representation of acne severity, which is why it was chosen as the basis for our study. Interestingly, a previous study demonstrated that training the evaluators was essential to significantly increase the reliability of the evaluation of lesion counting and the global assessment.⁹

Estimation of acne severity in the community is very challenging since it is difficult to explore the insight of the general population. Patient self-assessment is useful in clinical practice to obtain subjective information of acne severity before and during follow-up after acne treatment. In particularly now, during and after the coronavirus disease-19 pandemic telemedicine has become an effective option however there are limitations to virtual and photographic examination.¹⁰ Accurate self-assessment by patients could thus help in teledermatology management and follow-up. Self-reported acne severity may also be a potential tool in clinical trials and epidemiologic studies involving large number of subjects. Yet this type of instrument for patient self-evaluation of acne has not yet been developed, especially in Thailand. Therefore, we first aimed to validate an acne severity grading self-assessment suitable for the Thai population. Our secondary objective was to evaluate the benefits of training in improving the accuracy of this instrument.

Materials and Methods

For this cross-sectional study, a sample of university students from Mahidol University which is a government university located in central Bangkok, the capital of Thailand. Subjects were recruited on a voluntary basis. Our study was a pilot study, so the sample size was set at 50 subjects. Subjects provided written informed consent before enrollment. The study protocol was approved by the Mahidol University Institutional Review Board for Ethics in Human Research (MURA2016/822). We developed the Thai version of the acne severity grading self-assessments by translating and adapting the original version.^11,12 After reviewing the literature for available global assessments of grading acne severity we selected two apparently superior acne severity grading assessments: The Global Evaluation Acne Scale (GEA)^11,13 and Global Acne Grading System (GAGS).^12,14 (Table 1 and Table 2) The Global Evaluation Acne Scale (GEA)^11,13 was translated using a forward-backward technique with permission from an original author (Brigitte Dréno MD, PhD). The Global Acne Grading System (GAGS)^12,14 was also translated using a forward-backward technique with permission from an original publisher (John Wiley and Sons, Inc.). The translation process was conducted by the authors in collaboration with linguists from the Translation and Interpretation Center, Faculty of Liberal Arts, Mahidol University (TICLA).

Table 1 Global Acne Assessment Scale (GEA)

Table 2 Global Acne Grading System (GAGS)

In brief, two Thai translations of the original English version of the assessments were performed by two independent dermatologists. The translators compared both translations and formulated a consensus. Backward translation of the consensus version to the English version was performed by two independent native English speakers who were blinded to the original assessments. The backward translated assessments were reviewed for consistency with the original assessments by both dermatologists and the translators. If the backtranslations reflected any discrepancies in the meaning of the original English versions, the assessments were returned to the translators for further revisions. Then, to increase the suitability of the subjective self-assessment for the Thai population, minor adaptations with inalterability of the original meaning were performed by the authors. The Thai version of the GEA (TGEA) and Thai version of the GAGS (TGAGS) were finalized (Tables 3, 4 and Figure 1)

Table 3 Thai Global Acne Assessment Scale (TGEA)

Table 4 Thai Global Acne Grading System (TGAGS)

Figure 1 The six (I-VI) locations of Thai Global Acne Grading System (TGAGS).

Both of the Thai acne severity grading methods: TGEA and TGAGS were then further validated in participants with acne lesions. Subjects performed both self-grading methods in two rounds which were timed. During each round, the participants were asked to evaluate their acne severity by themselves using the TGEA and TGAGS in random sequencing of assessment methods. After completing the first round, participants were asked to rate the difficulty/ease of completing the TGEA and TGAGS. The scores were rated in a range from 0 to 10, where 0 indicated the most difficult and 10 indicated the easiest. After the first round of assessment, to minimize any recall that could affect the second round subjects were required to have a 30-minute rest interval period between each round. After the rest period before starting the second round, all participants received training about basic knowledge of acne and the different types of acne lesions. The sessions were conducted for approximately 15 minutes in small group interactive teaching (1–5 persons/class) with both lecture and illustrations. Then, the second self-grading of acne severity was performed as described for the first round. After both rounds of self-assessment by participants were complete, one designated dermatologist performed clinical assessment of acne severity using the TGEA and TGAGS to further validate the reliability between the subjects’ self-assessment and clinician assessment of acne severity using the TGEA and TGAGS.

Statistical Analysis

Descriptive statistics were computed for demographic characteristics. Reliability between the patient self-assessment and clinician assessment of acne severity was assessed by calculating the Cohen kappa coefficient (kappa) and intraclass correlation coefficient (ICC).

Cohen kappa coefficient (kappa) is a form of correlation coefficient used to test interrater reliability for nominal and categorical data. The kappa is range from −1 to +1 which ≤0 as indicating no agreement, 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement. While accepting 0.40 to 0.60 as “moderate” may refer the lowest value (0.40) is adequate agreement.^15,16

Intraclass correlation coefficient (ICC) is one of the most common techniques often used to evaluate the reliabilities for within-rater and between-rater of tools using continuous data research. ICC values approximating 1.0 indicate excellent reliability, values more than 0.81 indicated almost perfect reliability, and values ranging from 0.61 to 0.80 represent substantial reliability.^17,18

Both TGEA and TGAGS score were categorical data, we assessed reliability between the self-assessment and clinician assessment of acne severity by computing the kappa. Moreover, we tested the reliability between raw score of TGAGS of the self-assessment and clinician assessment by using ICC.

Continuous variables were compared to determine statistical differences. Mann–Whitney U-test was used to estimate the statistical significance difference of the duration of the TGEA assessment before and after the training session due to their non-nominal distribution. Whereas the Student’s t-test to assess the statistical significance differences of the duration of the assessment of the TGAGS and difficulty score before and after the training session due to their nominal distribution. Statistical significance in these tests was set at p ≤ 0.05. Data analysis was conducted using the statistical program Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP).

Results

Initially, a minimum sample size of fifty subjects was set, but seventy-seven Thai university students with acne vulgaris voluntarily enrolled. There were 56 (72.73%) women and 21 (27.27%) men, with an average age of 27.19 (±5.73) years. Demographic data of the study population are shown in Table 5.

Table 5 Demographic Data

TGEA

Thirty-seven (48.05%) participants who evaluated their acne severity had responses that corresponded to those of the clinician (kappa = 0.26, 95% confidence interval [CI] 0.22–0.30). The median duration of the assessment was 27 (8–79) seconds. The mean average difficulty score was 6.61 (±1.98). After receiving training, sixty-one (79.22%) participants who rated their acne severity had responses that corresponded to those of the clinicians (kappa = 0.66, 95% CI 0.65–0.71). The median duration of the assessment decreased to 13 (3–86) seconds (p < 0.001). The mean average difficulty score increased to 7.51 (±1.85) (p < 0.001; Tables 6, 7).

Table 6 Results of the Patient Self-Assessment Compared to the Clinician Assessment

Table 7 Time Used and Difficulty Score

TGAGS

Sixty (77.92%) participants who evaluated their acne severity had responses that corresponded to those of the clinicians (kappa = 0.52, 95% CI 0.40–0.57). The mean average duration of the assessment was 62.81 (±30.30) seconds. The mean average difficulty score was 6.82 (±1.97). After receiving the training, seventy-three (94.80%) participants who rated their acne severity had responses that corresponded to the clinicians (kappa = 0.89, 95% CI 0.88–0.92). The mean average duration of the assessment decreased to 31.48 (±15.70) seconds (p < 0.001). The mean average difficulty score increased to 7.97 (±1.56) (p < 0.001). According to the raw score of the TGAGS, the ICC during the self-assessment of acne severity compared to that of the clinician assessments was 0.54 (95% CI 0.36–0.68), and it increased to 0.79 (95% CI 0.69–0.86) after the training (Table 6 and Table 7).

Discussion

There are many different grading systems for the assessment of acne severity.¹³ The various grading systems have different assessment methods emphasizing on two main methods: acne lesion counting and photographic standard comparisons. Still, no grading system is yet considered to be the global standard. An ideal grading system should be accurate, reliable, simple and quick to perform. Lesion counting although would provide the best results for objective acne evaluation is tedious, while global grading scales would be more suitable in clinical settings, epidemiological and clinical research. Subject self-assessment provided by the patient can be used to explore the severity of acne in the general population. Another advantage of self-reporting is that the researcher can directly receive insight from the person experiencing the symptoms. It is also useful to recruit and assess subjects in acne trials and research. However, self-evaluation of acne severity has shown variable results.^19–27 Prior to this study there has been no data on self-reported acne severity in Thailand.

The results from this study, in the validation of the acne severity grading self-assessment, poor to moderate reliability of self-assessments was found in the first round because no specific educational training was given to the self-raters. However, we demonstrated that increasing knowledge among the raters by a short period of training clearly helped in reducing the discrepancies between subject self-assessment and the clinician assessment. Thus, after training sessions, the TGEA was classified as having substantial agreement (kappa = 0.66). Whereas the TGAGS was classified as having almost perfect agreement (kappa = 0.89). Assertion by the raw scores showed that after training the TGAGS also had a strong agreement (ICC = 0.79) that represented an almost excellent agreement between the TGAGS patient’s self-assessment and clinician’s assessment. Regarding the time duration required the for assessment, the TGEA and TGAGS were performed quickly. Concerning the level of difficulty, participants reported that the TGEA and TGAGS were easy to use. Several studies have shown a variable correlation of subjective self-assessed skin severity measures and clinician assessed objective measures of acne severity.^19–27 Most authors mentioned that the self-evaluated method is an unreliable approach to assessing the severity of dermatological conditions, as it has moderately poor agreement between self-assessment and clinician assessed acne evaluations.^19–23 A prior study reported fair agreement of acne self-evaluation by adolescents and adults compared to a dermatologist assessment, so the authors concluded that the self-evaluation of acne was inaccurate and insufficient for clinical or research.¹⁹ Other previous reports have also described a low to moderate agreement between self-diagnosed diseases and the clinicians’ diagnoses of acne, which showed a significantly lower subjective prevalence than the actual prevalence.^20,21 On the contrary, our current study showed an almost excellent reliability of the severity grading between subjects self-assessment and physicians. Such a finding was in accordance with those of other validation studies in patients with acne.^24–27 A prior study mentioned that the self-assessment was appropriate for determining the severity of acne.²⁰ Multicenter research has demonstrated good agreement between physician and patient global assessment of acne severity.²⁵ A good correlation between subjective complaints and objective signs of acne was found, indicating that the patient and the dermatologist agreed well on the severity grading of acne.²⁶ The high degree of concordance between patient self-assessment and clinician assessment in our study could be explained by our providing training session for our subjects. However, our participants were university students. We infer that both training and the degree of education are imperative to augment the accuracy of acne severity assessment.

Accordingly, a study validating the acne severity self-assessment showed that subjects’ experience and knowledge about acne and the treatment of acne aided in increasing the agreement between the acne self-assessment and clinician assessment.¹⁹ Similarly, Lucky et al²⁸ described the high variability between raters appears to be reduced by standardized training, which supported our results in that training improved agreement between the raters. Moreover, they also suggested the use of a template that divides the face into segments may help improve reliability of the assessment. Anyhow, Tan et al⁹ estimated reliability of acne lesion counting and global acne assessment by dermatologists, which also found that training enhanced reliability in both lesion counts and global assessment. Furthermore, Hayashi et al established an acne severity classification in Asian patients which is based on three different methods for grading, including general impression of dermatologists, photograph-based estimation by independent experts, and grading by lesion counting. They also found a higher reliability of the global assessment when assessors received training on all three assessment procedures.²⁹ To the best of our knowledge, no patient training was provided in previous acne self-assessment literatures. Our study is the first study which provided an educational training session for the subjects to augment the reliability of self-reported assessment of acne severity. Therefore, based on our study results and those of previous literatures, self-assessment may be a reliable tool for measuring the severity of acne, but additional training to assure basic knowledge of acne is needed to increase the consistency between subjects’ and the clinician’s diagnosis.

The potential limitations of our study were the small sample size and the homogeneous population with similar educational background and adult acne age group, which does not represent the entire acne-affected population. This may limit the generalizability of the results to other research or clinical setting. Further studies are needed to validate the use of this instrument in other patient groups.

Conclusion

In conclusion, the TGAG was superior to TGEA due to the almost perfect reliability between subjects’ and physician’s assessments. Thus, we suggested that the Thai translated version of GAGS maybe a reliable patient self-assessment of acne severity suitable for the Thai population. However, an educational training session is essential in enhancing the reliability of this instrument to ensure optimal accuracy. Applying this tool to clinical practice; in both telemedicine and onsite clinics, can help doctors gain insight of acne severity from the person experiencing acne and therapeutic response. The Thai translated version of GAGS can be accurately utilized by experts and non-experts and thus should be useful in epidemiological, community-based research and clinical trials as it is practical and low cost.