Development and Validation of a Patient-Reported Outcome Measure for Fingernail and Toenail Conditions: The NAIL-Q

Introduction

Nail conditions are common in the population, affect people of all ages, and have a broad differential diagnosis.^1–7 Nail conditions can cause distress due to their cosmetic appearance, symptoms such as pain, reduced function and mobility, reducing one’s health-related quality of life (HRQL).^8–11 Patient reported outcome measures (PROMs) are valuable tools that can be used to measure these concepts from the patient perspective.¹² PROMs are increasingly used in research studies to inform clinical care and improve health care quality.^13–18

There are currently seven published PROMs for nail conditions.^19–25 Five PROMs were developed for onychomycosis, including the Onychomycosis-Specific Questionnaire,¹⁹ Health-Related Quality of Life Measure for Onychomycosis,²⁰ Onychomycosis Disease-Specific Questionnaire (ODSQ),²¹ NAILQoL,²² and the OnyCOE-t.²³ Two PROMs were developed for nail psoriasis, including the Psoriasis Quality of Life Scale (NPQ10)²⁴ and the Nail Assessment in Psoriasis and Psoriatic Arthritis QOL (NAPPA-QOL).²⁵ A review that applied Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines^26,27 concluded that these seven PROMs had limited evidence supporting validity, reliability, and responsiveness.²⁸

A new PROM is needed to provide evidence about the impact and effectiveness of treatments for all types of nail conditions from the patient perspective. The aim of our study was to develop the NAIL-Q for use with patients with any type of fingernail or toenail condition using the Rasch Measurement Theory (RMT) approach.^29,30

Patients and Methods

Ethics Statement

Ethics board approval was obtained from the Hamilton Integrated Research Ethics Board approvals #7370 and #14622 and from Weill Cornell Medicine (New York, NY, USA).

Approach

We used a mixed methods approach³¹ and adhered to best practices guidelines for PROM development.¹²

Phase 1: Qualitative

In phase 1, Interpretative Description³² was used to generate knowledge relevant to a clinical context. Participants were recruited from a dermatology clinic in Canada and one in the USA and provided with a thank-you gift card for each task they helped with (ie, initial, cognitive, and final review), as described below. Research staff were asked to recruit a purposive sample of English-speaking adults who varied by age, gender, ethnicity, and nail conditions. Informed consent was obtained, and interviews were conducted by phone by a skilled qualitative researcher. A semi-structured interview guide was used and updated as new concepts were elicited (Table S1).

Interviews were audio-recorded, transcribed verbatim et literatim, and coded using a line-by-line approach. Concepts were labelled with a domain and major/minor theme using constant comparison to refine categories and form a conceptual framework.³³ Interviews continued until redundancy of concepts was reached.³⁴ Codes were used to develop items to form NAIL-Q scales. Scales were assigned instructions, a time frame for answering, and response options.

A series of cognitive debriefing interviews with patients were performed by phone/Zoom by two skilled qualitative interviewers. The think aloud method^35,36 was used to determine if participants found the instructions, response options, and scale content easy to understand, relevant, and comprehensive.

Interviews were audio-recorded, transcribed verbatim et literatim, and coded. The scales were revised based on patient feedback and included in a REDCap survey³⁷ to which dermatologists with nail expertise were invited by email to review. To establish content validity, experts were asked to identify any content that they deemed would be hard to understand, not relevant to nail conditions, or missing.

To ensure changes suggested by experts were acceptable to patients, a REDCap survey was sent to the cognitive debriefing sample via email. Participants were asked to review the scale and identify any items that were hard to understand, not relevant or missing. Nonrespondents were emailed after one week, followed by a phone call one week later. Feedback was used to finalize the NAIL-Q field-test version.

Phase 2: Quantitative

Data for this phase were collected from Prolific Academic (www.prolific.co), an online crowd working platform. Participants were paid the equivalent of 10.50 sterling per hour. A short pre-screen survey was directed to Prolific participants fluent in English. Instructions asked people to complete the pre-screen if they had a fingernail and/or toenail condition for 3 months or longer for which they had seen a health professional in the past. Six images depicting common nail conditions were included, and people were asked to identify their nail condition from the images, or to indicate “other” or “not sure” (Figure S1).

Participants with any of the six nail conditions were invited to complete the full survey. This survey included demographics (age, gender, race, marital status, education, daily activity, and country) and clinical questions, including severity, sides and number of nails affected, and how long the condition had lasted. Single items asked about pain, embarrassment, and activity interference. Participants were asked when they last saw a healthcare professional for their nail condition, and the number of appointments in the past 2 years. For seven treatments (topical medications, oral medications, biologics, laser, surgery, taping, injections, and clipping nails) participants were asked to indicate their status (ie, do not need, need, currently having, have had, had and need more). Those who had treatment were asked when they had treatment, and if the treatment made their condition better, the same, or worse.

For the NAIL-Q, participants whose nail condition affected fingernails and toenails completed the Appearance, Distress and Symptoms scales twice. The Outcome scale was completed by participants who had or were having treatment. The NAIL-Q was followed by six evaluation questions with disagree/agree response options: (1) asked questions in a respectful way, (2) was easy to understand, (3) was thorough, (4) asked questions that are important to me, (5) should be used by doctors treating people with nail conditions, and (6) would help my doctor better understand my nail condition. Finally, participants completed the generic EQ-5D-5L health-related quality of life utility PROM, which was scored using the USA normative values.³⁸ EQ-5D-5L includes 5 items (mobility, self-care, usual activities, pain/discomfort, anxiety/depression) with five response options that range from one (no problem) to five (unable to/extreme problem).

At the end of the survey, participants were asked to participate in a test–retest (TRT) study. Late respondents (less than seven days from the initial survey) and those with missing scale data were not invited. We asked if there had been a change in their nail condition and if treatment had been initiated since the initial survey. We aimed to include 100 participants per scale.

The RMT analysis used RUMM2030 software and the unrestricted Rasch model for polytomous data (RUMM version 2030, RUMM Laboratory Pty Ltd, Duncraig, Western Australia, Australia, 1998–2021). Our goal was to identify the best subset of items to retain in each scale. The RMT analysis is described in detail elsewhere.²⁹ Briefly, we performed a range of statistical and graphical tests to examine if the data fit the Rasch model. We adjusted the sample size to 500 for tests that used a X² analysis. We examined: (1) item response categories to determine if item responses were properly ordered; (2) item fit statistics to determine if scale items mapped out an item difficulty continuum; (3) local dependency to determine if item residuals were too closely related to each other and performed subtests if residual correlations were ≥0.20; (4) targeting to determine how well a scale measured the experience of the sample; (5) Person Separation Index and Cronbach's alpha values³⁹ to determine scale reliability using criteria that values should be ≥0.70;^26,40 and (6) Differential Item Functioning (DIF) to determine if items in a scale behaved the same for age (<30 vs ≥30 years), gender (man vs woman), country (UK vs USA), education (completed college/trade school or university yes/no), and location (toenail vs fingernail). DIF analysis was performed with random samples to ensure subgroups were equivalent in size and repeated three times. We only performed DIF if there were at least 150 participants in the analysis. Any items with DIF were split on the characteristic, and person locations from the original and split analyses were correlated to inspect the impact of DIF on scale scoring.

In SPSS Version 28, we performed Principal Component Analysis and hypothesized that scale items would load ≥0.70 providing evidence that the items were part of the latent variable.⁴¹ Scale scores were transformed from 0 (worst) to 100 (best) based on person locations. For the TRT data, anyone who reported their nail condition had changed, or they had initiated treatment, were excluded. Intra-class correlation coefficients (ICC) were computed with a two-way random effects model; values should be ≥0.70.^26,40

For convergent validity, NAIL-Q scales were correlated with the EQ-5D-5L scores. COSMIN criteria stipulate that the correlations should be ≥0.50 between scales measuring similar constructs, 0.30–0.50 between scales measuring related but dissimilar constructs, and <0.30 between scales measuring unrelated constructs.²⁶ We hypothesized that with EQ-5D-5L, the NAIL-Q’s Appearance and Outcome scales represent dissimilar constructs, and the NAIL-Q’s HRQL scales represent related but dissimilar constructions. In terms of construct validity, we tested six hypotheses of known group differences (Table 1).

Table 1 Construct Validation Hypotheses and Results

Parametric and nonparametric tests were used based on whether the scale data were normally distributed assessed by Skewness and Kurtosis values.⁴² Categorical variables with small cell sizes were combined for analyses. P-values of <0.05 were considered statistically significant.

Results

Phase 1: Qualitative Study

Interviews with 19 women and four men were conducted between October 2019 and November 2020. Table S2 shows sample characteristics.

Participants described a range of outcomes that mattered to them. Participants were concerned about having weak nails that chip, break, crack, split, peel, loosen, and fall off [“Well they are also splitting and cracking and peeling off”]. Pain was a common symptom [“I will get an ingrown nail on my finger, and it swells up. It gets painful, extremely painful”]. Some participants described trouble using their hands [“I need to remember that, oh, I need to grip that differently or hold that differently”], and feet [“…I started to feel it every time I was walking”]. Nail appearance was typically described negatively [“unattractive”, “ugly”, “not normal”, “bad”]. Specific problems included pits, indents, or nails that were uneven, thin, or discolored (eg, spots or stripes). Participants generally disliked their nails [“It was terribly embarrassing”], and participants felt self-conscious of their nails [“…when I used to take the train, holding onto a pole, I’m conscious then of it…you know, being messed up”]. Many participants described concealment behaviors [“I keep my polish on my nails all the time. If I don’t, I put Band-Aids over them”].

The first draft of NAIL-Q included six scales with 89 items. Changes made in each round are shown in Table S3 and summarized in Table S4. Revisions took place between June 2021 and January 2022. Round 1 included seven cognitive debriefing interviews. Based on this round, 16 items were revised, 20 items dropped, and two items added. NAIL-Q had 71 items after Round 1. In Round 2, 35 clinical experts were invited to review the NAIL-Q and 11 (five men, six women) complied. Experts were from the USA (n = 5), Canada (n = 4), Brazil (n = 1) and Switzerland (n = 1). All experts were dermatologists; five had intermediate and six had advanced expertise in nail conditions. Based on this round, two items were revised, one item dropped, and 13 items added. At the end of Round 2, NAIL-Q had 83 items. In Round 3, the seven cognitive interview participants were invited to review the NAIL-Q and five participated. Based on this round, one item was revised, and one item added. The final field-test version included 84 items.

Phase 2: Quantitative Study

The field-test study took place in June and July 2022. The recruitment process is shown in Figure 1. The screen, full survey, and TRT took participants 3 (SD = 2.4), 12.8 (SD = 6.4) and 8.5 (SD = 5.7) minutes on average to complete, respectively.

Figure 1 Recruitment flow diagram.

Table 2 and 3 show the sample demographic and clinical characteristics, respectively. The 555 participants came from 23 countries and ranged in age from 18 to 80 years (Mean 36.7; SD = 13.9). Toenail conditions were more common (n = 441) than fingernail conditions (n = 186). A small sample (72, 13%) had both fingernail and toenail conditions. Most participants (n = 325, 58.6%) had seen a healthcare professional about their nails 1 or 2 times in the past 2 years.

Table 2 Sample Demographic Characteristics for 555 Participants

Table 3 Sample Clinical Characteristics

The RMT analysis reduced the NAIL-Q to 45 items. Figure 2 shows the final conceptual framework. Each aspect of the framework is measured by a NAIL-Q scale. The scales include five or six response options that measure satisfaction (Nail Appearance), frequency (Nail Distress, Physical: Fingernails, Physical: Toenails), concern (Nail Symptoms; Strength: Fingernails), and agreement (Treatment Outcome). Item-level fit statistics and DIF results are shown in Table S5. All 45 items had ordered thresholds and non-significant X² P values after Bonferroni adjustment. Item fit was within +2.5 for 37 items. DIF was identified for 14 items including 2 of 35 tested for country, 4 of 45 tested for age, 4 of 35 tested for gender, and 9 of 23 tested for location. Pearson correlations between person locations for items before and after item split for DIF indicated marginal impact of DIF on scoring, with all correlations ≥0.996.

Figure 2 Framework for the NAIL-Q.

Scale-level results are shown in Table 4. The data fit the Rasch model for six scales with a slight misfit for Strength: Fingernails (p = 0.02). PSI values were ≥0.79, and Cronbach's alpha values were ≥0.83 with and without extremes. Three pairs of items in the Appearance scale and one pair in the Outcome scale evidenced local dependency. Subtests to examine their impact on reliability showed a drop in PSI values of ≤0.01. The proportion of patients to score on the scale ranged from 76.5% to 96%. While two scales evidenced some ceiling effect, it should be noted that 80% (Physical: Fingernails) and 83% (Physical: Toenails), of those to score at the ceiling chose “Not at all” when asked: “How much does your nail condition interfere with doing your usual daily activities?” Targeting can be seen graphically in the person-item threshold distributions (Figure S2). The item thresholds distributions (blue histogram) were spread over a reasonable range that matched the sample (pink histograms).

Table 4 RMT Scale-Level Statistics and Other Scale Findings

TRT data for toenails (n = 95) and fingernails (n = 97) provided 192 assessments for the Appearance, Distress, and Symptoms scales, 140 for the Outcomes scale, 95 for the Physical: Toenails, and 97 for the Physical: Fingernails and Strength: Fingernails scales. TRT participants who reported a change in their nail condition (n = 12) or started a treatment (n = 18) were excluded. The ICC values for all NAIL-Q scales were ≥0.81, which exceeded the COSMIN criteria (see Table 4).

Principal Component Analysis supported a single factor for all 7 scales, with factor loadings ≥0.78 to 0.94. Pearson correlations between NAIL-Q scales and the EQ-5D-5L scores met the COSMIN criteria for five scales Appearance (r = 0.18, n = 615), Distress (r = 0.31, n = 614), Physical: Fingernails (r = 0.30, n = 182), Physical: Toenails (r = 0.40, n = 433), and Outcome (r = 0.10, n = 427). The two remaining scales had correlations that were weaker than hypothesized (Symptoms, r = 0.26, n = 615; Strength: Fingernails, r = 0.10, n = 182).

Table 1 summarizes the construct validity findings, and Tables S6–S11 provide the detailed results. All 6 hypotheses were supported by the data. As nail condition severity increased, NAIL-Q scores were incrementally lower (see Table S6). Those with a nail condition that affected both sides reported lower scores than those whose nail condition was on one side (see Table S7). As pain increased, symptom scale scores decreased (see Table S8). Participants who reported feeling more embarrassed about their nail condition reported worse NAIL-Q scores (see Table S9). Additionally, as interference with activities increased, scores on the NAIL-Q scales decreased (see Table S10). Finally, following treatment, scores on the Outcome scale were higher for those who reported their nail condition as better compared with those whose condition was the same or worse following treatment (see Table S11).

Finally, Table 5 shows the proportion of participants to agree/disagree with the six evaluation questions. Most participants agreed that the NAIL-Q asked questions in a respectful way, was easy to understand, was thorough, asked important questions, should be used by doctors treating people with nail conditions, and would help doctors better understand nail conditions.

Table 5 Participants to Disagree/Agree to NAIL-Q Evaluation Questions

Discussion

Nail disorders are common and comprise approximately 10% of all dermatological conditions.⁴³ The NAIL-Q is a rigorously developed and tested PROM that can be used in research and clinical care to measure outcomes that matter to people with nail conditions. The NAIL-Q was developed using a patient-centred approach in line with guidelines for PROM development.¹² Our analysis refined a set of independently functioning scales with strong psychometric properties. The evaluation questions provided additional evidence of content validity by showing the NAIL-Q was easy to understand, asked relevant and important questions in a respectful way, and should be used to inform clinical care.

A recent review assessed the content measured by seven nail-specific PROMs.^19–25,44 Most items in existing PROMs measure physical and psychological concerns.⁴⁴ While the NAIL-Q includes such concerns, it also covers additional concepts, including a scale to measure satisfaction with nail appearance. In clinical trials of treatments for nail conditions, how the nail looks (eg, how thick, clear, even, healthy it looks) represents a proximal outcome important to patients but is largely overlooked in existing PROMs.⁴⁴ Although, five of the seven existing nail-specific PROMs measure nail appearance,^{19–21,23,25} only the onychomycosis-specific PROM from Lubek et al²⁰ and the OnyCOE-t™²³ provide a separate score for this construct.^28,44 While such a concept could be measured using objective measures, today it is important in clinical trials to also measure outcomes from the patient perspective.⁴⁴ In addition to outcomes, measuring patients’ experience of care is integral to providing patient-centered care. Previously developed nail-specific PROMs include only a limited number of items that address treatment outcomes,⁴⁴ with only the psoriasis-specific NAPPA-PBI and onychomycosis-specific Lubek et al²⁰ instrument scoring this construct.²⁵ The NAIL-Q, in contrast, includes a scale that can be used post-treatment for any nail condition to evaluate satisfaction with nail treatment (eg, pleased with result, worth the time/effort, would recommend to others, better than expected). We expect this scale will provide useful feedback to physicians and researchers about different nail treatments from the patient perspective.

An important advantage of the NAIL-Q is that its scales were designed to be used across all nail conditions, with most scales applicable to both fingernails and toenails. Other nail-specific PROMs were designed for use in psoriasis and onychomycosis.^28,44 Using standardized PROM measurements across nail conditions will allow comparability of results between conditions and across studies and could also help facilitate the implementation of PROMs into clinical practice for purposes such as benchmarking, value-based payment, and shared decision-making.^18,45–47

Limitations

Our phase 1 study included only four males. However, this limitation was addressed in our phase 2 study, which included more gender diversity; 43.8% identified as male, and 2.3% identified as nonbinary. Furthermore, the DIF analysis by gender showed that items with DIF had negligible impact on scoring. The qualitative sample to develop the NAIL-Q only included people from Canada and the USA. The field-test sample, on the other hand, was open to anyone fluent in English on the Prolific platform. While a strength of the study was the testing of NAIL-Q in people from multiple countries, a limitation is that we were only able to examine DIF by country for the UK and USA. Additional research to explore the psychometric performance of the NAIL-Q in countries where our sample was limited in size is warranted. Another limitation is that since the phase 2 sample was community-based, it was not possible to confirm nail diagnoses clinically, nor the severity ratings, as both variables were self-reported. Future research could examine the psychometric performance of the NAIL-Q in a clinic-based sample with confirmed diagnoses and objective measures of nail severity. Future research to measure responsiveness (ability to measure change) and to determine minimally important differences for NAIL-Q scales is also needed.

There are many advantages to recruiting research participants using online platforms such as Prolific. Such studies can accrue large diverse international samples quickly and at low cost. Online platforms are ideal for TRT studies as it is possible to control the collection of retest data on a specific day and for a specific number of participants. While crowdsourcing platforms may be a useful means to engage research participants, there are limitations. Many factors can affect data quality, including lack of attention, comprehension, honesty, and reliability.⁴⁸ We used Prolific as it has been shown to provide high-quality data compared to other similar platforms.⁴⁸

Conclusion

The NAIL-Q is a rigorously designed nail-specific PROM developed using a modern psychometric approach.¹² The NAIL-Q can be used to measure outcomes for adults with any fingernail or toenail condition. The NAIL-Q is available on the following website: www.qportfolio.org/nail-q/.