Validity of Self-Reported Diagnosis of Osteoporosis in Japan Nurses’ Health Study

Introduction

Osteoporosis is a disease characterized by low bone mass and micro-architectural deterioration of bone tissue leading to enhanced bone fragility and a consequent increase in fracture risk.¹ Self-reporting is frequently used to assess subjects’ disease status in cohort research. The cost-effectiveness and feasibility of this methodology make it an attractive approach in countries without national disease registries, such as Japan. Nevertheless, the unreliability of self-reported information is a serious problem because it can introduce error into epidemiological investigations of risk factors. This is especially true for the new incidence of osteoporosis in a cohort. When evaluating a patient’s osteoporosis history, healthcare providers must bear in mind that an affirmative response in a questionnaire is not necessarily equivalent to a definitive medical diagnosis because patients may be falsely remembering or even hiding some aspects of their health history. Ideally, their answers should be corroborated against their medical records; usually, however, these records cannot be acquired for all members of a cohort.

Another problem is that the clinical diagnosis of osteoporosis includes the presence of a fragility fracture and/or low bone mineral density (BMD) of ≥2.5 standard deviations (SDs) below the mean for a young adult means (YAM) without fracture.¹ However, because it is not feasible to perform BMD measurement in all participants, case definitions of osteoporosis in large-scale epidemiological studies and national health surveys often rely on self-reports of diagnoses, medications, or fractures.^2–4 Further complicating the interpretation of self-reported data is the fact that the validity of such data can depend on background factors, such as ethnicity, as well as cohort-specific characteristics.

The Japan Nurses’ Health Study (JNHS) is an ongoing nationwide prospective cohort study of >15,000 female nurses, begun in 2001, to ascertain how women’s health is affected by lifestyle factors, healthcare practices, and physical status over their lifetime.⁵ The Gunma Nurses’ Health Study (GNHS) is a preceding pilot study, begun in 1999. The health outcomes assessed include the incidences of different cancers, cardiovascular diseases, osteoporosis, and female-specific diseases such as endometriosis and leiomyoma. The fact that these cohorts are composed exclusively of female nurses has yielded valuable insights into disease incidences and risk factors unique to women in the profession, making it superior to general-population surveys in that regard.

Although the validity of self-reported osteoporosis is often questioned, validation studies are lacking. In the present study, we investigated the validity of self-reported diagnoses of osteoporosis in young and middle-aged women in the JNHS and GNHS cohorts.

Patients and Methods

Subjects and JNHS/GNHS

The JNHS is an ongoing prospective cohort study being performed to investigate the effects of lifestyle and health care practices on women’s health in Japan. Detailed information on its design, population, protocol, and sample size calculations have been published previously.^5,6 Briefly, the baseline (BL) survey was conducted from 2001 to 2007, with a planned follow-up of 30 years. In total, 15,019 women agreed to participate in the follow-up, and they signed and returned the informed consent form along with the completed survey. The study population comprised female licensed nursing professionals, such as registered nurses, licensed practical nurses, public health nurses, and midwives, who were ≥25 years of age in the JNHS and ≥20 years of age in the GNHS. All resided in Japan at the time of the BL survey. Questions on this survey regarded personal information, occupation, physical characteristics, smoking and alcohol consumption, dietary habits, reproductive health history, use of female hormone agents, use of other drugs and supplements, personal and family history of disease, and the results of any medical examinations. Follow-up is currently ongoing; once every 2 years, the subjects are mailed a similar self-administered questionnaire to complete and return by post.

Prior to the JNHS, the feasibility of its research strategy and the validity of its questionnaires were investigated and confirmed in the GNHS, a pilot cohort study that was begun in 1999 (n = 698).^7,8 We combined the JNHS and GNHS datasets for the present work (n = 15,717), hereinafter called the JNHS. Table 1 shows the number of subjects in each age group. The women had a mean age at BL of 41.6 ± 8.3 [median (Q1–Q3), 41 (35–48)] and a mean follow-up period of 11.5 ± 4.4 years [median (Q1–Q3), 12 (10–16)].

Table 1 Numbers of Subjects in Each Age Group at Baseline

The JNHS Coordination and Data Center is located in the Epidemiological Research Office of the School of Health Sciences at Gunma University. The studies were conducted in accordance with the Declaration of Helsinki. The GNHS study protocol was approved by the institutional review board of Gunma University, Japan (Approval No. 3, 1999), and the JNHS study protocol was approved by the institutional review board of Gunma University, Japan (Approval No. 101, 2001) and by the ethics review board of Japan’s National Institute of Public Health, Japan (Approval No. 03007, 2003). Informed consent was obtained from all individual participants included in the study.

Data Collection and Corroboration (Figure 1)

In the BL and biennial regular follow-up questionnaires, women were asked, “Have you ever been diagnosed with osteoporosis by a medical doctor?” If they answered “yes,” they were asked to provide their age at first diagnosis. We identified and isolated those women who self-reported the incidence of osteoporosis at BL and during regular follow-up by June 2018. To corroborate the self-reported positive cases, an additional confirmation questionnaire was sent to these women who affirmed osteoporosis during regular follow-up. The subjects were again asked the same question as above; they were additionally asked to provide details about their age at diagnosis, how osteoporosis was detected, the method of examination, young adult mean (YAM) % of BMD, and their history of treatment with drugs for osteoporosis. This clinical information was furnished to an expert review panel composed of osteoporosis specialists to verify each self-reported positive diagnosis. In this study, diagnostic circumstances only (DCO) refers to the women with osteoporosis who did not self-report osteoporosis but instead self-reported diagnostic circumstance evidence such as fragility fracture or use of drugs for osteoporosis.

Figure 1 Flow diagram illustrating data collection and corroboration in Japan Nurses’ Health Study.

Diagnosis of Osteoporosis in This Study

Primary osteoporosis is diagnosed when no diseases causing low BMD other than osteoporosis and no secondary osteoporosis are observed and the results of bone assessment meet the following requirements by the Diagnostic Criteria for Primary Osteoporosis in Japan: 2012 Revision,⁹ followed by the World Health Organization diagnostic criteria for osteoporosis.¹⁰ In this study, women who met the criteria in I-A, I-B, or II were diagnosed with primary osteoporosis:

I-A. Presence of a fragility fracture

1. Presence of a fracture in either the lumbar spine or proximal femur

2. Presence of another fragility fracture and a BMD below 80% of YAM

I-B. Absence of fragility fracture

BMD is ≤70% or −2.5 SD of YAM. Quantitative ultrasound is not included as part of the diagnostic criteria for osteoporosis.

II. Use of drugs of effectiveness grade A for BMD in the guideline for the prevention and treatment of osteoporosis (Table 2).¹¹ Conjugated estrogens are excluded for women aged 20 to 44 years because such drugs are mainly used to treat ovarian insufficiency in these patients.

Table 2 Drugs with Grade A Effectiveness for Bone Mineral Density in the Guideline for the Prevention and Treatment of Osteoporosis

Women who were diagnosed with primary osteoporosis were also considered to have secondary osteoporosis if they met the following criteria:

S-A: Use of anti-estrogenic drugs

S-B: Use of steroid hormones

S-C: Endocrinologic or metabolic disease affecting bone metabolism excluding diseases for which drugs listed in S-A and S-B are used (eg, rheumatoid arthritis, premature menopause)

In this study, women who self-reported osteoporosis at BL and during regular follow-up but were not diagnosed as positive or negative by the expert panel were considered to have an indeterminate diagnosis.

Validation

These three sources (BL, regular follow-up, and confirmation questionnaire) were validated in terms of their positive predictive value (PPV) and negative predictive value (NPV) for the cumulative incidence of osteoporosis. For the first two sources, the validation sample included all members of the study cohort (n = 15,717). The PPV of BL and regular follow-up was calculated as the number of verified positive cases of osteoporosis divided by all self-reported cases at BL and during regular follow-up. The NPV was calculated as the number of suspected negative cases divided by all members of the validation sample who did not self-report osteoporosis at BL and during regular follow-up. The suspected negative cases comprised all members of the validation sample for osteoporosis in question, minus (A) cases who self-reported osteoporosis at BL and during regular follow-up and (B) positive cases whose status was established by DCO.

The PPV of BL, regular follow-up, and the confirmation questionnaire was calculated as the number of verified positive cases of osteoporosis divided by all cases who corroborated their positive diagnosis on the confirmation questionnaire. The NPV was calculated as the number of suspected negative cases divided by all members of the validation sample except those who self-reported their positive diagnosis on the confirmation questionnaire. The suspected negative cases comprised all members of the validation sample minus (A) cases who self-reported their positive diagnosis on the confirmation questionnaire, (B) cases ruled positive by DCO, and (C) contradictory cases (ie, women who were confirmed by expert review but who self-reported a negative status in the confirmation questionnaire, left the field blank, or did not send the confirmation questionnaire). After the osteoporosis cases were fixed, the cumulative incidence of osteoporosis at the age of 40, 50, 60, 70, and 80 years was estimated using the Kaplan–Meier method.

Results

Verified Cases of Osteoporosis

A flow diagram illustrating the validation process of osteoporosis is shown in Figure 2. The number of cases of self-reported osteoporosis at BL and during regular follow-up was 1189, and 729 of these cases were verified by expert review. Some subjects (n = 1105) who sent the confirmation questionnaire corroborated their positive diagnosis (n = 720) but were eventually ruled negative by the expert panel (n = 104), and the incidence was verified by the confirmation questionnaire and expert review in 616 subjects. Conversely, the percentage of subjects who indicated a negative diagnosis on their confirmation questionnaire was 24.1% (266/1105).

Figure 2 Flow diagram illustrating the validation process of osteoporosis and the results. Abbreviations: OP, osteoporosis; DCO, diagnostic circumstances only. Notes: 1) Including 11 deaths, 69 withdrawals, and 4 cases of missing data. 2) Including 10 cases who gave a response other than “positive” or “negative.” 3) Diagnosis by baseline and regular follow-up questionnaires.

In the JNHS cohort (n = 15,717), the number of DCO cases was 155; the estimated numbers of total osteoporosis cases, primary osteoporosis cases, and secondary osteoporosis cases were 884, 812, and 72, respectively; and the estimated incidence rates of osteoporosis were 5.6%, 5.2%, and 0.5%, respectively. The cumulative incidence of osteoporosis at the age of 40, 50, 60, 70, and 80 years was estimated to be 0.1% (95% confidence interval, 0.1–0.2), 1.1% (0.9–1.3), 7.7 (7.0–8.4), 23.6% (21.6–25.7), and 54.2% (40.2–68.1) by the Kaplan–Meier method, respectively (Figure 3).

Figure 3 Cumulative incidence of osteoporosis in Japan Nurses’ Health Study.

Self-Reported PPV/NPV for Osteoporosis

Table 3 summarizes the PPVs and NPVs of the two sources [(i) BL and regular follow-up + expert review and (ii) BL and regular follow-up + confirmation questionnaire + expert review] for the incidence of osteoporosis. The BL and regular follow-up + confirmation questionnaire achieved higher PPVs than BL and regular follow-up alone (85.6% vs 61.3%, respectively). Self-reporting achieved NPVs close to 100% for osteoporosis for both (i) BL and regular follow-up and (ii) BL and regular follow-up + confirmation questionnaire (98.9% vs 98.2%, respectively).

Table 3 Self-Report PPV/NPV for Osteoporosis (Validation Samples =15,717)

Discussion

Self-Reporting and Additional Confirmation Questionnaire

To determine the validity of the self-reported incidence of osteoporosis in our cohort, we investigated the PPV of self-reported positive diagnoses as affirmed at BL and during regular follow-up and as corroborated by an additional confirmation questionnaire. Self-reporting at BL and during regular follow-up achieved a PPV of 61.3% for osteoporosis. One study indicated that the validity of self-reporting is associated with individual characteristics,¹² and our cohort consisted entirely of nursing professionals. While some evidence suggests that the validity is negligibly affected by educational attainment, we partially attribute the high accuracy of self-reporting in the present study to the uniformly high level of medical education and deeper knowledge of osteoporosis in our cohort than the general population.¹³ In Australian women, the self-reported prevalence of osteoporosis compared with osteoporosis definitions based on self-reported medication and Pharmaceutical Benefits Scheme prescription reimbursement claims showed a PPV of 29.2% and NPV of 99.5% in middle-aged women (56–61 years of age) and 57.1% and 94.8%, respectively, in older women (79–84 years of age).¹⁴ In the present cohort, self-reporting at BL and during regular follow-up plus the confirmation questionnaire achieved higher PPVs (85.6%) than self-reporting at BL and during regular follow-up alone (61.3%). Another study showed that self-reported dual-energy X-ray absorptiometry (DXA) results underestimated the prevalence of osteoporosis.¹⁵ Our investigation leads us to conclude that the incidence of osteoporosis cannot be accurately assessed by self-reporting in isolation, making corroboration by additional questionnaire and ruling by an expert panel necessary. It seems that self-reporting in isolation fails to capture the real incidence of osteoporosis, even for this cohort of nursing professionals with uniformly high medical knowledge, and an additional confirmation questionnaire to confirm the details is mandatory.

Incidence of Osteoporosis in This Study

In our cohort, the estimated incidence rates of total osteoporosis, primary osteoporosis, and secondary osteoporosis were 5.6%, 5.2%, and 0.5%, respectively, and the cumulative incidence of osteoporosis at the age of 40, 50, 60, 70, and 80 years was estimated to be 0.1%, 1.1%, 7.7%, 23.6%, and 54.2%, respectively. In an epidemiological study of osteoporosis in Japanese women, the prevalence of L2–4 osteoporosis in the age groups of ≤39, 40–49, 50–59, and 60–69 years was 0.0%, 1.9%, 5.3%, and 13.5%, respectively, and that of femoral neck osteoporosis was 0.0%, 2.9%, 4.8%, and 22.2%, respectively.¹⁶ Thus, the estimated incidence rates of osteoporosis in this study are appropriate for Japanese middle-aged women.

Fractures could occur also in osteopenic or normal BMD women with some fracture risk factors (eg family history, associated diseases) and clinical evaluation should consider also clinical risk factors. We will plan to consider the association between the risk factors and the incidence of fracture in JNHS.

Strengths of This Study

One of the strengths of the present study is the large sample size. Our sample size (n = 15,717) was much greater than those reported in similar validation studies.^14,17 The large population-based sample and the small differences in characteristics of women who were excluded from the current analyses support the generalizability of the results. In addition, we believe that our data are more reflective of the general Japanese population than past findings in other regional cohorts to date because geographical variation is minimized by the nationwide scope of the JNHS. Moreover, our cohort was relatively homogenous in terms of sex and occupation, consisting entirely of female nursing professionals.

Limitations of This Study

However, the present study also has some limitations. The main limitation of this study is the lack of a true gold standard for the diagnosis of osteoporosis, namely bone densitometry. However, given the population-based design of the study and the fact that more than half of the sample lived in rural and remote areas where availability of DXA scans is scarce,¹⁸ exposing all women to a DXA scan is neither a feasible nor cost-effective use of resources. However, osteoporosis is well known to be an underdiagnosed and undertreated condition.^18–20 The current reference standards were based on medication use and thus could not identify women with osteoporosis who did not use anti-osteoporosis medication. In this study, we diagnosed osteoporosis as (1-A) the presence of a fragility fracture or (I-B) low BMD according the diagnostic criteria for primary osteoporosis in Japan,⁹ or the use of drugs of effectiveness grade A for BMD according to the guideline for the prevention and treatment of osteoporosis¹¹ by the BL and regular follow-up and confirmation questionnaires. The cohort-specific characteristics may also limit the generalizability of our findings, especially the participants’ relatively young ages.

The JNHS database covers the whole of Japan. The cohort has a relatively homogeneous background, consisting of female nursing professionals with a uniform level of medical education. This means that this dataset should produce highly reliable findings if analyzed for risk factors for osteoporosis (such as taking hormone therapy and lifestyle factors) and their effects. In addition, the specialists responsible for verifying the incidence of osteoporosis in the present study were blinded to the patients’ exposure to risk factors. For this reason, we believe that subsequent investigations of risk factors using this dataset will not be affected by information bias. We plan to continue our work by analyzing the respective contributions of different risk factors among confirmed cases of osteoporosis, as verified above.

Conclusion

We confirmed the PPVs and NPVs of the self-reported incidence of osteoporosis in our cohort and found that self-reporting was associated with high NPVs for the cumulative incidence of osteoporosis but with slightly lower PPVs. Additional corroboration by confirmation questionnaires could improve the PPVs.