Convergent Validity of Three Pressure Injury Risk Assessment Scales: Comparing the PPRA-Home (Pressure Injury Primary Risk Assessment Scale for Home Care) to Two Traditional Scales

Introduction

Geriatric care is a growing concern for several industrial nations, especially given the growth in the elderly population paired with societal depopulation (fewer births relative to deaths).¹ In fact, in 2018 it was estimated 28% of Japanese citizens were over age 65, and the proportion is expected to reach approximately 38% by 2065.² In light of this situation, the Japanese government has promoted a healthcare focus shift from acute care to home- and community-based care provided in home settings.^3,4 In order to provide comprehensive community-based care, the Japan-specific geriatric care system (the Community-based Integrated Care System, a medical-social welfare networking model) works with the Japan-specific long-term care insurance (LTCI) system.^2,5 Under the LTCI system, various comprehensive healthcare services are implemented (medical, nursing, preventive care, and daily living supports). Gatekeeping for this system is performed by care managers who are certified as social welfare professionals and possess the specialized knowledge necessary to help individuals in need of nursing care achieve more independent lives.^6,7 The care manager is responsible for providing a variety of care management services for individuals, including checking their physical condition and living circumstances, drawing up and adjusting care plans according to their needs, and monitoring their care. To obtain a care manager license, an individual must have five or more years of geriatric care experience as a medical professional (registered nurse or medical doctor) and/or a social welfare professional (care worker or social worker) and pass the national examination.⁸ Unfortunately, care managers from the social welfare discipline may not possess sufficient knowledge or practice regarding the prevention and treatment of pressure injuries.

Most individuals in the geriatric population have one or more chronic disease associated with increased risk of pressure injury development: diabetes, heart disease, respiratory illness, and dementia.⁹ Pressure injury is a common chronic wound defined as a localized skin injury resulting from a combination of mechanical forces—friction, shear force, and pressure—over a prolonged period of time.¹⁰ These forces on the skin result in soft tissue deformation, followed by ischemic damage that leads to ulceration.¹¹ An increase in pressure injury prevalence in home healthcare settings is strongly associated with a wide spectrum of socioeconomic problems, including lower clinical quality indicators, compromised quality of life, and financial crises due to healthcare costs.^12,13 The pressure injury prevalence in Japan is reported to range from 3.6% to 10%.^14–16 Moreover, elderly individuals at living at home or within their community had worse pressure injury problems than those receiving care at an acute care hospital.¹⁷

Appropriate multidisciplinary collaboration among various professionals is required to provide quality pressure injury management during home care.¹⁸ Care managers have a key role, as they act as intermediaries between medical and social welfare professionals. As with many ailments, prevention is more effective than treatment in pressure injury management. However, Japanese care managers generally do not consult with physicians and nurses about pressure injury prevention prior to their home-care clients actually developing pressure injuries.¹⁹ This lack of communication likely is due to the fact that most care managers believe medical professional input and intervention is not needed until after a pressure injury has formed.

A pressure injury risk assessment scale is a powerful tool for promoting communication between care managers and the medical and social welfare professionals who serve home-care patients. The Braden scale for predicting pressure sore risk (Braden scale) is the most commonly used tool for predicting an individual’s risk of developing a hospital- or facility-acquired pressure injury.^20,21 The Ohura-Hotta (OH) scale also is widely accepted as a Japan-specific tool for predicting pressure injury risk in elderly individuals.²² The Japanese Society of Pressure Ulcers published a consensus statement on the OH scale’s efficacy based on clinical evidence from clinical trials with hospitalized patients.¹⁰

Despite the benefits of performing pressure injury risk assessments, care managers have reported having difficulty using the Braden and OH scales.¹⁹ These scales may be incompatible with home care because they were designed for use by staff at acute care hospitals. Additionally, acute care and social welfare professionals have distinct perspectives, not the least of which is their differing clinical environments.¹⁹

Recently, a newly developed risk assessment scale targeted to Japanese care managers, the Pressure Injury Primary Risk Assessment Scale for Home Care (PPRA-Home), has been investigated.²³ The PPRA-Home is expected to be successfully applied through a multidisciplinary collaboration that includes care managers who act as a bridge between medical and social welfare professionals. However, few studies have evaluated the validity of the PPRA-Home.²³ To address this research gap, this study compared the PPRA-Home’s convergent validity with that of the standardized Braden and OH scales.

Methods

Study Design and Setting

This multicenter, cross-sectional study was conducted with 34 different home-based geriatric support service providers located in the Japanese cities of Kiryu (Gunma prefecture), Tenri (Nara prefecture), Sendai (Miyagi prefecture), Yamagata (Yamagata prefecture), and Hikone (Shiga prefecture). The study period was from November 2019 to June 2020. Other researchers have conducted similar studies. Methodology that convergent validity study of the use of participants with versus without injury using risk screening tools at one point in time has been reported by Seongsook et al.²⁴ They performed a convergent validation testing aimed at pressure injury prevention by comparing the Braden scale with other scales in hospitalized patients with and without pressure injuries. Additionally, Morita et al conducted a similar validity study using the PPRA-Home among participants with long-term care needs at a Japanese geriatric health services facility.²³ Overall, the current study was designed in accordance with these previous studies.

Sample

Eligible study participants were individuals in the five cities who were classified at care levels 1 through 5, as defined in previous reports.^6,25 Briefly, the LTCI system’s medical and social welfare public insurance program is available to individuals aged 40–64 years who have certain diseases and those age 65 and over. These individuals are divided into 7 categories: assistance levels 1 and 2, and care levels 1 through 5. Individuals at assistance level 1 and 2 do not require medical/nursing care for daily living activities, but some assistance is occasionally required in order to prevent the need for support of medical/nursing care. Compared to individuals who require assistance level 1, those who require assistance level 2 seem to slightly decline in activities of daily living. They could be potentially assigned in the near future from assistance level to care level. Individuals at care level 1 require partial support for daily living, while those at care level 5 have difficultly living at home without extensive assistance from medical and/or social welfare professionals. The maximum benefits individuals covered by the LTCI could receive for in-home services were determined according to their care need level. Individuals were excluded from the study if they could not or did not give written informed consent to participate in the study.

The sample size was based on a power of 0.8 and an alpha of 0.05, assuming the area under the curve (AUC) for the PPRA-Home and the other two standardized scales to be 0.8 and 0.9, respectively. Therefore, a sample of at least 29 participants per group (pressure injury group and no pressure injury group) was needed for this study. Participant recruitment for the study ended the day enrollment reached the planned sample size.

Instruments

Three pressure injury risk assessment scales were used in this study: the PPRA-Home, Braden, and OH scales. The PPRA-Home was generated in the Japanese language in 2019.²³ According to the developer, the PPRA-Home’s items were determined after reviewing elderly Japanese individuals’ risk factors for pressure injury development. Care managers can easily use the tool’s simple question and answer format in their daily work. The PPRA-Home includes eight items, each requiring a yes or no response: spontaneous body turning, lean body type (or not), mobility, reduced food intake, skin moisture, diaper use, edema, and use of head-of-bed elevation. Assessors assign a score of 1 for yes responses, and 0 for no responses. Scores for each item are summed for a total score ranging from 0 to 8, with higher scores indicating a higher risk for developing a pressure injury.

The Braden and OH scales served as comparison tools. The Braden scale includes six subscales: sensory perception, moisture, activity, mobility, nutrition, and friction/shear. The friction/shear subscale is scored from 1 to 3 and the remaining subscales are scored from 1 to 4. Lower scores indicate greater risk. The lowest total score is 6, indicating the highest risk for developing a pressure injury. The appropriate cutoff score for determining risk depends on the patient population and clinical settings.²⁶ A recently published retrospective study indicated that the Braden scale cutoff score in a Japanese hospital with long-term care ward was 14, with moderate diagnostic accuracy.²⁷

The OH scale consists of four subscales: spontaneous body turning, sacral bony prominence, edema, and articular contracture. Higher scores indicate greater risk. Although the OH scale developer did not provide a cutoff score for determining risk, the total score provides three categories of risk level for pressure injury development: low risk (1–3), moderate risk (4–6), and high risk (7–10).²² The clinical application of an algorithm, which chooses a support surface for each patient based on the OH scale, reduced the incidence of pressure injury development in a Japanese general hospital.²⁸

Procedures

PPRA-home assessors had a care manager license as well as experience working in the home-based care field and serving as a home-based geriatric support service provider in the districts where this study was conducted. Braden and OH scale assessors were physicians or expert nurses certified in wound, ostomy, and continence (WOC) care. These professionals had worked in either an acute care hospital, a long-term care hospital, or a clinic near the facilities where the care managers worked. The investigators explained the study outline to the assessor candidates and received their written agreement to participate in the study.

A pair of assessors (one care manager and one physician or WOC nurse) performed the data collection during the care managers’ regular visits at participants’ homes or day service centers. The day service centers where data collection took place are facilities where individuals requiring long-term care can receive assistance completing the activities of daily living (ADL; eating, bathing, functional training, and physical/mental rehabilitation). Among the two assessors for each participant, one (care manager) supervised the participant in actual practice under the LTCI system; therefore, the assessors using instruments were able to know the participants’ condition and whether they had pressure injury or not prior to the start of this study. While two assessors visited each participant’s home or day service center at the same time data were collected, each performed an independent assessments without communicating with the other assessor. Care managers were not trained to use the PPRA-Home prior to start of this study, because the researchers wanted to assess the tool’s validity under the care manager’s current level of knowledge and experience. The physicians and WOC nurses did not require training on the Braden and OH scales because, as wound care specialists, they all had extensive experience using the scales in their respective institutions. The number of times of measurement by each assessment scale was one per participant, due to the cross-sectional nature of this study. The assessments took an average of two minutes per assessor to complete.

In order to compare the PPRA-Home accuracy with that of the other two scales, study participants were divided into two groups: those with a pressure injury and those without. The assessor (physician or WOC nurse) visually confirmed the presence or absence of a pressure injury at the time of assessment. When a pressure injury was observed, the assessor judged its stage using the Japanese standard DESIGN-R^® scoring system in which d1 represents non-blanchable erythema; d2 a lesion extending to the dermis; D3 a lesion extending into the subcutaneous tissue; D4 a lesion extending into muscle, tendon, and bone; D5 a lesion extending into an articular or body cavity; and DU indicates that depth measurement is impossible.²⁹

Data Collection

All data obtained from this study were recorded on written data collection forms. The records were reviewed for the assessor’s demographic data (age, sex, years of care manager experience [only for care managers]), and previous occupation [only for care managers], and the participant’s demographic data (age, sex, living arrangement, care level, functional disability level [bedridden level], and cognitive impairment level [dementia rating]). Bedridden and dementia level information was obtained from official data produced by the Care Needs Level Certification Committee from each city’s administrative office. The bedridden level was ranked from Rank J (requires partial support) to Rank C (remains in bed all the time and requires care for all ADL). The dementia rating ranged from independent (very minor cognitive disorder) to M (severe cognitive disorder). Item and total scores for each of the three scales also were recorded on the data collection form.

Statistical Analysis

For demographic data, the continuous variables are expressed as mean and standard deviation, while the categorical variables are expressed as frequencies and percentages. Each scale’s item and total scores were compared for the two groups prior to convergent validity analysis.

In the validity study, the correlations between the total scores for each of the three scales were investigated using the Pearson’s correlation test. Next, the diagnostic accuracy of the three tools was compared using receiver operating characteristic (ROC) statistics. Each scale’s level of accuracy for evaluating pressure injury risk was determined by calculating AUC with a 95% confidence interval (CI). An AUC value of 1 indicates perfect diagnostic accuracy, whereas a value of 0.5 or below indicates no more predictive accuracy than random chance (0.5 = 50/50 chance of being accurate or the same chance of accuracy as flipping a coin). For this study, an AUC value of >0.9 was considered excellent; >0.7 and ≤0.9 were considered moderate; >0.5 and ≤0.7 were considered less accurate; and ≤0.5 indicated the scale had no validity.³⁰ The PPRA-Home’s and the other scales’ AUC were compared using Delong’s test which could be used to show that the AUCs of two models significantly differ statistically.³¹ Sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and Youden index (sensitivity + specificity −1) also were employed to evaluate scale validity. The Youden index is a commonly used technique for defining an optimal cutoff value of each risk assessment scale on the ROC curve with the highest vertical distance from the 45 degree diagonal line.³² A p value of <0.05 denoted statistically significant differences. All statistical analyses were conducted using Statistical Package for the Social Sciences version 20.0 software (IBM Corporation, Tokyo, Japan).

Results

Assessor Characteristics

Data from 40 care managers and nine physicians/WOC nurses were collected and their characteristics are shown in Table 1. Regarding care managers, most assessors were 50–59 years old (48%) and female (75%). While about half of the assessors (54%) had a certified care worker license, only 13% had a nursing license (registered nurse and/or licensed practical nurse). Similar age and sex distributions was found among the sample of physicians and WOC nurses.

Table 1 Assessor Characteristics

Participants Characteristics

Of the 69 participants, 23 lived in Kiryu, 21 in Tenri, 12 in Sendai, nine in Yamagata, and four in Hikone. Thirty participants had a pressure injury and 39 did not. Among the 30 pressure injury participants, 10 were classified as d2, 15 as D3, one as D4, and the remaining four as DU. The demographic characteristics of the two participant groups are shown in Table 2. The mean age was 82.2 ± 9.6 in the pressure injury group and 85.3 ± 7.6 in the no-pressure injury group (p=0.16). Beside functional disability, there were no significant between-group differences in participant demographic characteristics. Participant’s characteristics of each pressure injury classification in pressure injury group are also represented in Table 3. Variation in patient’s characteristics such as age, sex, and level of disability was observed to be equivalent for each pressure injury classification in pressure injury group.

Table 2 Participant Characteristics

Table 3 Participant’s Characteristics in Each Pressure Injury Classification for Pressure Injury Group

Results of Three Risk Assessment Scores

Each tool’s item and total scores for both groups are shown in Table 4. For the PPRA-Home tool, total scores for the pressure injury group and no pressure injury group were 5.0 ± 1.8 and 3.2 ± 2.1, respectively (p<0.05). The average pressure injury group’s scores for the spontaneous body movement, mobility, and head-of-bed elevation items were significantly higher than were those of the no pressure injury group. However, no similar trends were observed in the remaining items.

Table 4 Three Risk Assessment Scores’ Ability to Predict Pressure Injury Development

Significant between-groups differences also were found in the Braden and OH scales’ averaged total scores. In the Braden scale, the pressure injury group’s scores were statistically significantly lower than those of the no pressure injury group for all items except nutrition. In the OH scale, the pressure injury group had a higher score than the no pressure injury group in every item except edema.

Correlation Between PPRA-Home and Other Two Scales

The correlation coefficient (r), decision coefficient (R²), and statistically significant differences between the PPRA-Home and the Braden and OH scales are shown in Table 5. While the PPRA-Home was negatively correlated with the Braden scale (r=−0.79, p<0.05), it was positive correlated with the OH Scale, with a statistically significant difference (r=0.58, p<0.05). The negative correlation observed between the PPRA-Home and Braden scale is due to the fact that the two scales are scored with opposite directions. In other words, low score in Braden scale indicates higher risk, and higher score in PPRA-Home and OH scale indicates higher risk.

Table 5 Pearson’s Correlation Coefficient for the Relationship Between PPRA-Home and Two Other Risk Assessment Scales

Convergent Validity

The AUC for the PPRA-Home was 0.737 (95% CI, 0.617–0.857; p<0.05), indicating that moderate accuracy could be obtained between the two study groups. Similar results were obtained with the Braden scale (AUC, 0.814; 95% CI, 0.711–0.917; p<0.05) and the OH scale (AUC, 0.794; 95% CI, 0.617–0.857; p<0.05; Table 6). According to Delong’s test, there were no statistically significant AUC differences between the PPRA-Home and the Braden and OH scales.

Table 6 Area Under the Receiver Operating Characteristic Curve for the PPRA-Home, Braden Scale, and OH Scale

The best cutoff score for each of the three scales was determined by simultaneously maximizing the Youden Index. On the PPRA-Home, a cutoff score of 4 had a sensitivity of 63.3%, specificity of 81.6%, positive likelihood ratio of 3.44, and negative likelihood ratio of 0.50. The cutoff value was determined to be 14 for the Braden scale and 6 on the OH scale (Table 7).

Table 7 Results from the Three Pressure Injury Risk Assessment Scales for Cutoff Score, Sensitivity, Specificity, Positive Likelihood Ratio, Negative Likelihood Ratio, and Youden Index

Discussion

Determining the validity for a newly developed pressure injury risk assessment scale is a key imperative, which ultimately could benefit individuals who receive preventive care as well as the caregivers who provide it. Although the PPRA-Home is targeted to the Japanese care manager in the context of pressure injury prevention in home care, its validity has not been adequately addressed in contemporary literature.²³ Kring et al reported that validity for the Braden scale could be divided into three types: content validity, convergent validity, and predictive validity.²⁶ In this study, we addressed the convergent validity of the PPRA-Home in order to investigate this tool’s correlation with other similar scales. The results of the current study revealed a moderate correlation between the PPRA-Home and the Braden and OH scales. Furthermore, all three scales had similar AUCs, suggesting that care managers in real-world clinical settings should consider using the PPRA-Home tool. In Japan, the Braden and OH scales have been widely used as established pressure injury risk assessment scales in actual clinical settings (especially in acute care institutions where wound care specialists belong); however, their application to home care setting is still insufficient. Researchers reported that the current guidelines for pressure injury prevention could be incompatible with home care because the guidelines were based on care algorithms that were produced by clinical cases in acute care settings.³³ Currently, a majority of individuals who have care manager’s license in Japan are non-clinician, and more specialize to home care setting. It is considered that the busy schedule made it difficult for care managers to acquire the knowledge/practice and correctly use of the Braden and OH scales in their daily work.³⁴ Therefore, the use of the PPRA-Home is expected to contribute for improving the quality of pressure injury assessment in home care.

The most significant finding was the fact that the PPRA-Home’s AUC as assessed by care managers was similar to that of the Braden and OH scales as assessed by physicians and WOC nurses (Table 6). The Braden and OH scales are considered gold standard tools for evaluating a patient’s pressure injury development risk, and as such have been used to develop a pressure injury preventive program widely used in acute and long-term care hospitals.¹⁰ Both the Braden and OH scales had an ideal sensitivity and specificity score of 1 on the AUC analysis, an almost impossible achievement. Kring et al reported that a sensitivity of 75% and higher is reasonably robust finding in a validity study for such tools.²⁶ While the PPRA-Home’s sensitivity was below 75%, the scale’s validity still is considered acceptable because it produced an AUC similar to those of the other two standardized scales.

In the PPRA-Home, an averaged score for the spontaneous body movement, mobility, and head-of-bed elevation items showed significant differences between the pressure injury group and the no pressure injury group (Table 4). However, these differences were not found for the remaining items. This result might be explained by the terms used in the PPRA-Home and the interpretations of each item by the assessors and participants. Because care managers are responsible for checking physical conditions and living circumstances, the PPRA-Home developer used terms and definitions that enable the care managers to report information from their visual observations of and interviews with the individual in their care.²³ For example, the term using diapers was intended to detect earlier pressure injury risk from reduced skin barrier function due to fecal incontinence-associated dermatitis. In fact, this study found diaper use itself is not a significant risk factor for pressure injury development. Additionally, because the reduced food intake, skin moisture, and lean body type items involve subjective evaluations, care manager assessments may vary. Currently, the authors consider the PPRA-Home a pressure injury risk assessment scale that should be available to Japanese care managers for use in home-care settings. In order to further investigate the tool’s efficacy, the authors will examine the PPRA-Home’s content validity in a subsequent study.

These results regarding cutoff scores for the Braden and OH scales may prove valuable to healthcare professionals who use these scales in daily clinical practice. The cutoff value for the Braden scale was 14, which aligns with the findings from a previous retrospective study conducted in a long-term care ward at one Japanese hospital.²⁷ In contrast, although the OH scale developer divided scores into three levels of a risk for developing a pressure injury, the best cutoff score has not yet been determined for this scale.²²

To complete the validity investigation for the PPRA-Home, its predictive validity must be examined. Previous studies have examined the Braden scale’s predictive validity for various types of patients, ranging from surgical patients to nursing home residents.^35,36 However, prospective and/or retrospective cohort analysis will be required to be able to predict how many individuals will develop pressure injury based on their PPRA-Home total score. Due to the cross-sectional nature of this study, the authors did not address a predictive validity study which could completely predict the development of pressure injury using a risk screening tool such as PPRA-Home. The predictive validity testing should be conducted in a longitudinal study design. Future studies on predictive validity testing will clarify whether the cut-off value estimated in this study is optimal or not. Recommend studying subjects at baseline and then several months later and then seeing how scales predict risk of injury (ie, difference in scores with those having injury versus those without at several months later). The authors believe that our current study is a vital step for assessing the validity of PPRA-Home and that the cut-off value of this study can potentially use as an indicator of predicting pressure injury development in a home care setting.

Determining the best cutoff scores for pressure injury risk assessment scales could help identify high-risk individuals who should receive preferential prophylactic intervention. While various intervention techniques for hospitalized patients have already been well established (support surface, nutrition, prophylactic dressing, and skin cream), few have been reported for individuals requiring long-term care at home.^37–40 Therefore, a future investigation regarding the development of preventive intervention program based on the PPRA-Home assessment is needed.

There was a limitation in this study. As the authors performed a power calculation to estimate the appropriate sample size, a sample size of >29 participants per group (pressure injury group and no pressure injury group) would be sufficient to achieve the objective to just this study. However, a further longitudinal approach involving a large sample size with a rigorous subject selection criteria will be required to completely assess the validation of the PPRA-Home.

Conclusions

This multicenter, cross-sectional study was conducted to investigate the convergent validity of the PPRA-Home for pressure injury prevention in home care. The PPRA-Home AUC as assessed by care managers was similar to AUCs for the Braden and OH scales as assessed by physicians or certified WOC nurses. ROC analysis indicated that a total score of 4 was the best cutoff score for the PPRA-Home. More research is needed to establish the PPRA-Home’s content and predictive validity. To address clinical needs, in future research the study authors will develop a Japan-specific pressure injury preventive program based on the PPRA-Home.