Differences in Estimates of Post-Herpetic Neuralgia Between Medical Chart Review and Self-Report

Introduction

Herpes zoster (HZ) is a viral infection caused by the reactivation of latent varicella zoster virus, typically presenting as a painful rash along a single dermatome. The most common complication of HZ is post-herpetic neuralgia (PHN), a debilitating condition of intense and often treatment-resistant pain that can persist for years after the initial HZ diagnosis.¹ PHN can profoundly impact patients’ quality of life^2,3 and increase health-care utilization.⁴ Estimates of PHN vary widely, ranging from 5% to 30%.^5–8 In addition to differences in study design, patient characteristics, PHN definition, and HZ vaccination status, the approach used to assess pain likely impacts these estimates as well, although this has not been well examined.

Accurately assessing pain is important for epidemiological estimates of HZ and PHN, as well as for studies of vaccine effectiveness; however, pain can be difficult to quantify. Existing research has relied on several methods to evaluate PHN. Although there are advantages to each, there are also distinct opportunities to introduce error. Programming approaches, such as electronic algorithms using diagnosis codes and medications from electronic health records (EHR), are time-consuming to validate, and may over- or under-estimate cases that are not coded correctly.⁹ In addition, variation in treating chronic pain can make algorithm development difficult. Other approaches may increase sensitivity or specificity in the identification of PHN but are more labor-intensive. While manual review of EHR can provide a good estimate of health-care utilization related to pain, this approach is susceptible to human error and may underreport PHN cases among individuals who seek care less often.¹⁰ Although arguably the best assessment of an individual’s experience, self-reported pain is multi-dimensional and highly subjective and may be impacted by a variety of other factors, such as additional comorbidities, level of education, and quality of life.^11–13

For PHN, variation in study designs and sample populations makes it challenging to tease out how different approaches for measuring pain impact estimates of risk. Despite the widespread use of chart review and self-report for pain assessment, little research exists comparing outcomes obtained with each method. In an effort to help clarify this issue, we compared PHN risk assessed using both manual chart review and self-report.

Methods

The study was conducted at Kaiser Permanente Southern California, an integrated health-care delivery system serving over 4 million members. We calculated the risk of PHN among patients with laboratory-confirmed incident HZ between January 2012–February 2015 using both chart review of EHR and self-reported surveys. For laboratory confirmation, trained research associates collected at least two specimens of skin lesions per the protocol from the National Varicella Zoster Virus Laboratory at the Centers for Disease Control and Prevention, where standard polymerase chain reaction tests were performed.¹⁴ Patients were identified using EHR in a previous study examining the risk of PHN in HZ patients vaccinated and not vaccinated with zoster vaccine live (ZVL). Patients age ≥60 years were included with the exception of two younger patients who were selected as part of the matching process, described in detail elsewhere.¹⁵ Demographic information was obtained from EHR and face-to-face interviews conducted at baseline.

For self-report, the Zoster Brief Pain Inventory (ZBPI), a validated survey modified from the Brief Pain Inventory specifically to evaluate HZ- and PHN-related pain,¹⁶ was administered to study participants by phone interview at 90 days post-HZ diagnosis. One question asked patients to rate their worst pain in the past 24 hours on a scale of 1 to 10, with a higher number indicating more severe pain. A score ≥3 was considered to meet the operational definition of PHN.¹⁶

For chart review, EHR of medical visits occurring 90–180 days after the initial HZ diagnosis for the same sample population were reviewed by trained research associates for evidence of pain persisting at the site of the original HZ rash. Prior studies have used a variety of cut-off points to define PHN.^6,11,16-18 We opted to include visits occurring in the 90–180 day post-HZ diagnosis window in order to align with the administration of the ZBPI at day 90, as well as capture individuals who may delay in seeking care for symptoms. The following descriptors were included as pain-related symptoms: tingling, heat/burning sensation, irritation, discomfort, and paresthesia. Pain was classified as PHN if it was consistent with the HZ episode and could not be explained by other obvious causes. Because of variation in provider documentation, pain severity could not be measured. A diagnosis of PHN made by a physician or attached to a prescription medication was also included as a PHN case. Recurrent HZ cases (where HZ symptoms resolved and subsequently returned) during the 90–180-day window were not considered to be PHN.

For our analysis, we excluded any patients missing ZBPI assessments. The study was reviewed and approved by the KPSC Institutional Review Board. For the original recruitment study, all participants provided written informed consent. A waiver of informed consent was granted for this study.

Analysis

Descriptive statistics were calculated for the study population. To assess differences in PHN ascertainment with each method, we cross-tabulated frequencies of PHN identified from chart review with self-report. To further evaluate how each approach differed in subsets of the population, we also stratified these analyses by sex, age group (<70, ≥70 years), and ZVL vaccination status. Due to the age distribution of our study participants, we used 70 years as the cut-off point for our comparison of younger and older groups. Analyses were conducted using SAS 9.3 (SAS Institute, Inc, Cary, North Carolina).

Results

General Characteristics

Among the 1018 patients who were included in the study, 37 were unable to be reached to administer the ZBPI and were excluded from the study. The final sample population comprised 981 patients, including 616 females (62.79%) and 365 males (37.21%), and there were 314 participants (32.01%) age <70 years and 667 (67.99%) age ≥70 years (Table 1). Multiple racial/ethnic groups were represented, including White (n=558, 56.88%), Hispanic (n=205, 20.90%), Asian (n=99, 10.09%), and Black (n=70, 7.14%). The most common level of education attained was a bachelor’s degree or above (n=370, 37.72%). Most of the participants were retired (n=767, 78.19%). The majority of patients were married/living with partner (n=664, 67.69%).

Table 1 General Characteristics of the Study Population

Chart Review

Using our chart review criteria, we found a PHN risk of 7.34% (n=72) (Table 2). We identified a higher proportion of females (n=52, 8.44%) with PHN compared to males (n=20, 5.48%) (Table 3). Between the age groups, PHN varied by a small percentage (<70 years: n=19, 6.05%; ≥70 years: n=53, 7.95%), and was comparable among vaccinated (n=38, 7.58%) and unvaccinated (n=34, 7.08%) individuals.

Table 2 Post-Herpetic Neuralgia (PHN) Identified with Chart Review and the Zoster Brief Pain Inventory

Table 3 Post-Herpetic Neuralgia (PHN) Identified with Chart Review and the Zoster Brief Pain Inventory, by Sex, Age Group, and Zoster Vaccine Live (ZVL) Vaccination Status

Self-Report

At 90 days post-HZ diagnosis, 12.23% (n=120) of the study population met the operational definition for PHN (Table 2) from the self-report survey. PHN risk was relatively similar between sexes, identified among 11.36% (n=70) of females and 13.70% (n=50) of males (Table 3). The percentage of individuals with PHN from self-report was almost twice as great among those age ≥70 years (n=95, 14.24%) compared to age <70 years (n=25, 7.96%). PHN risk among those who had received ZVL vaccination (n=46, 9.18%) was also substantially lower than among unvaccinated individuals (n=74, 15.42%).

Chart Review and Self-Report

The two approaches were largely in agreement, with 89.20% (n=875) concordance. While chart review yielded lower estimates of PHN than self-report for the overall study population, as well as for the sex-, age-, and vaccination status-subsets, we also found some differences between the two approaches. Among the 981 participants, 29 (2.96%) PHN cases were identified through chart review but not self-report, and 77 (7.85%) were classified as having PHN from self-report, but evidence of PHN was not found from chart review (Table 2).

Although PHN risk from self-report was slightly higher among males than females (13.70% and 11.36%, respectively), it was lower from chart review (5.48% and 8.44%, respectively). The difference between PHN risk identified with the two methods was notably smaller among females (2.92%) than males (8.22%).

In the younger group, PHN risk identified with each method varied only by 1.91% (chart review: 6.05%, self-report: 7.96%), whereas the discrepancy was substantially greater (6.30%) in the older population (chart review: 7.95%, self-report: 14.24%).

The difference in PHN risk detected with each method was minimal (1.60%) among vaccinated individuals (chart review: 7.58%, self-report: 9.18%). In the unvaccinated group, however, the discrepancy between the two approaches was much more pronounced (8.33%; chart review: 7.08%, self-report: 15.42%).

Discussion

Our two approaches to evaluate PHN risk provide insight about the variation found in existing literature and highlights the ways in which each may capture different outcomes. This comparison is important for determining the best approach to assess PHN in future work, such as vaccine effectiveness studies, evaluating new treatment modalities for PHN, or calculating related health-care costs.

We found notable discrepancies in PHN risk estimates ascertained with self-report versus chart review, with approximately 5% greater risk identified from self-report. A previous study conducted among cancer patients also noted a substantial difference between self-reported symptoms and chart review, with the prevalence of pain obtained from patient surveys almost twice that of chart review.¹⁹

PHN detected with chart review is likely influenced by health-seeking behavior.²⁰ Although a higher percentage of males reported pain ≥3 on the ZBPI compared to females (13.70% and 11.36%, respectively), a smaller proportion had a PHN-related visit (males: 5.48%, females: 8.44%). This is aligned with prior research indicating more frequent health-care utilization among females.²¹ We also found that PHN estimates obtained with each approach were much closer among the younger age group (6.05% chart review, 7.96% self-report) than the older group (7.95% chart review, 14.24% self-report), which could potentially be related to additional barriers to seeking medical attention among older individuals, such as mobility or transportation issues.²² Furthermore, previous research has reported that health-care utilization tends to be higher among vaccinated individuals.¹⁰ Our results support this as well, with marked discordance between the two methods among unvaccinated participants (chart review: 7.08%, self-report: 15.42%), and a much smaller difference among vaccinated individuals (chart review: 7.58%, self-report: 9.18%).

Several additional factors may explain the disagreement between the results detected with each approach. While the ZBPI defined PHN as pain severity ≥3, our chart review considered any mention of pain (or the related keywords) persisting at the original site of the HZ rash as PHN, irrespective of pain level,¹⁶ as this was not consistently captured in EHR. Therefore, PHN identified by the chart review would not necessarily meet the ZBPI pain score used in our operational definition of PHN. In addition, our chart review captured cases where a prescription for pain medication was attached to a PHN diagnosis, whereas pain that was well controlled by medication could have been missed by the ZBPI. It was also possible that pain that was initially manageable with medication at the time of the ZBPI administration stopped being effective at a later date or that patients who experienced no relief from prescription medication or other treatment modalities could have believed that further care was futile and opted not to see their doctor. Furthermore, while pain was solicited in the self-report approach, chart review depended on the patient mentioning pain and the provider documenting it accordingly. Variation in day-to-day pain might also explain why chart review captured certain PHN cases that the ZBPI missed, which has been previously noted in research.¹⁶ Finally, the different timeframes used for each approach might have contributed to discrepancies.

There are some limitations to consider when interpreting our results. While the ZBPI appears to capture PHN more comprehensively than chart review, there are possible sources of error which should be noted. As with all self-reported measures, the potential for non-response bias exists (ie, patients who agreed to participate in the study may have differed from non-participants). For the original study, 47.64% of eligible patients contacted declined participation (45% of vaccinated, 56% of unvaccinated).¹⁵ Among the 1018 patients included in this study, 37 (3.63%) could not be reached for the survey administration at 90 days post-HZ diagnosis. We did not investigate patterns of missingness and are unable to determine if this had any effect on our results. In addition, despite the short timeframe asked about in the ZBPI (past 24 hours), respondents may experience recall error.²³

Finally, additional factors that might affect health-seeking behavior could impact the measurement from chart review and further affect the concordance between the two approaches.

Conclusion

Our findings highlight the value of each method in the assessment of specific endpoints related to long-term pain following HZ. While the ZBPI appears better suited to capture the burden of illness experienced by the patient, administering these surveys is not feasible for large, observational studies, and there is potential for non-response bias. EHR are available for chart review on all members. However, chart review can potentially underestimate PHN risk—particularly among unvaccinated individuals—which would thus affect HZ vaccine effectiveness estimates, likely due to differences in health-care utilization among certain subpopulations. Therefore, adjusting for variables indicative of health-seeking behavior, such as prior utilization, is essential.