Agreement Between Self-Reported Information and Administrative Data on Comorbidities, Imaging and Treatment in Denmark – A Validation Study of 38,745 Patients with Knee or Hip Osteoarthritis

Introduction

In research and in clinical practice, self-reported information on the medical history of the patient is often preferred over medical records or registry-based data because it is more accessible and associated with lower cost.^1,2 However, self-reported information may be affected by recall bias and social desirability bias,¹ and the accuracy of the answers can also be influenced by the design of the specific questionnaire and the specificity of the questions.³ Haapanen et al showed that diabetes, hypertension, coronary heart disease and general cardiovascular diseases reported by the participants were highly compatible with what was reported in medical records. On the contrary, the information taken from the participants about musculoskeletal problems, eg low back problems, knee and hip osteoarthritis (OA), was incompatible with the information in medical records.⁴

OA is a degenerative joint condition that causes joint pain, functional problems, reduced quality of life, and it is one of the leading causes of pain and disability globally.^5,6 Since accurate information on the medical history (eg comorbidities, medication, and previous surgery) is important for decisions on the treatment of OA,^5,6 it is important to determine the accuracy of self-reported information obtained from patients with OA. To evaluate the accuracy of self-reported information in different conditions, previous studies have compared the self-reported information with medical records,^4,7–10 administrative databases^11–14 or clinical examinations.^15,16 Although none of these methods can be considered as the gold standard, administrative databases covering large populations, available at all times and with a high level of completeness and validity can be considered as one of the most useful and accurate ways of validating self-reported information.^17,18 To our knowledge, there are no previous studies which investigate the agreement between commonly used self-reported information obtained from patients with OA and registry-based data with national coverage.

Good Life with osteoArthritis in Denmark (GLA:D) is a nationwide program that aims to implement treatment guidelines for knee and hip OA in clinical practice using self-reported and objectively measured information.¹⁹ The aim of our study was to examine the agreement between self-reported information (comorbidities, radiographs, knee and hip surgery, prescription-based pain medication) obtained from patients with knee or hip OA participating in GLA:D and data obtained from the nationwide Danish registries.

Materials and Methods

Design

This was a registry-based study using baseline and follow-up data from 38,745 patients with symptoms and/or functional limitations of the knee or hip OA participating in GLA:D with data from the Danish national registries.¹⁹ This study was reported according to recommendations of the STROBE statement for reporting observational studies.²⁰

Ethics and Permissions

According to the North Denmark Region, Local Ethics Committee, ethical approval was not needed for GLA:D program. The Danish Data Protection Agency at SDU has approved the GLA:D registry (approval no.: SDU; 10.084) and the current analyses (approval no.: SDU; 10.124). According to the Danish Data Protection Act, patient consent was not required as personal data was processed only for research and statistical purposes. This study was approved by Statistics Denmark’s Scientific Board.

Participants

GLA:D consists of 2 sessions of patient education and 12 supervised, group-based exercise sessions led by a physiotherapist over an 8-week period. Participants referred to GLA:D from the initiation of the program in January 2013 until the end of December 2018 were included in this study. There are 3 ways for a patient with OA to get referred to GLA:D in Denmark: 1) by their general practitioner, 2) by themselves or 3) by an orthopedic surgeon. Participants were evaluated for eligibility by the treating physiotherapist. Participants understanding Danish with symptoms and/or functional limitations of knee or hip OA were eligible if they did not have any other reasons for their joint symptoms than OA (eg inflammatory joint disease or patellar tendinopathy) or other symptoms that were more prominent than their OA symptoms (ie pain in other body regions). According to Danish²¹ and international guidelines;⁵ radiographs are not needed to diagnose OA,⁵ and therefore not part of the eligibility criteria for GLA:D. Patient characteristics, details of the GLA:D treatment program and outcomes have previously been published.¹⁹

Variables

Self-Reported Information by Patients

All the self-reported data were available from the GLA:D registry. Self-reported data was collected from patients through online questionnaires distributed via email. Education and pain intensity (reported on a 0–100 visual analogue scale (VAS) as average pain during the previous month) as well as comorbid diseases (collected from 01.07.2014, category of diabetes was divided into Diabetes type 1 and Diabetes type 2 after 26.08.2018 and high cholesterol was only collected from 24.10.2018) were reported by the patients at the baseline evaluation. The comorbid diseases were divided into 14 categories and patients reported the presence of each condition as yes or no. At baseline and at 12 months follow-up, patients self-reported whether they underwent hip replacement surgery (yes/no) and knee replacement surgery (yes/no) prior to GLA:D or during the 12 months follow-up, respectively.

Self-Reported Information Collected by Physiotherapist

Information on age and gender was extracted from the personal identification number, unique to each person living in Denmark (CPR number). Whether or not a radiograph was taken (no, yes, more than 6 months ago, yes, within the last 6 months, do not know) was questioned by the treating physiotherapist at the baseline evaluation, while BMI was calculated from height and weight entered by the physiotherapist. Information about previous surgery for the right/left knee and hip joints (yes/no) was questioned at baseline evaluation (collected from 13.05.2016). If the patient reported having previous surgery on either hip or knee, the type of surgery was also obtained for that specific joint. Patients reported the type of surgery from a list of 15 surgery subgroups for knee and 13 surgery subgroups for hip. Pain medication type was obtained from the patients by the treating physiotherapist at the baseline evaluation. Examples of the different medications were given in the GLA:D data collection. Patients reported the use of medication within the last three months from a list of 12 medication categories.

Patients and physiotherapists type in their response to questions directly in the online GLA:D^® registry system. No paper forms are used. The database manager routinely performs a data validation procedure checking continuous variables for extreme values and categorical variables for unexpected distributions. In case of deviations error detections are performed.

National Data Sources

Three national data sources were used to examine the agreement between self-report and registries: the National Patient Registry,²² the National Prescription Registry,²³ and the Danish Civil Registration System.²⁴ The CPR number, assigned to all Danish residents, was used to link all data sources.²⁴ The Danish National Patient Registry (DNPR) contains nationwide data on all inpatient admissions since 1978 and outpatient and emergency room contacts since 1995. Also, since 2003 it covers private hospitals as well as private specialty clinics except private practice specialists and general practitioners (GP). Treatments (surgery and other treatments) and examinations (radiological and other examinations) are registered by the physician after the completion of procedure. Surgical operations are coded according to the Nordic Medico-Statistical Committee (NOMESCO). For each patient contact, one primary (main reason for the hospital contact) and relevant secondary discharge diagnoses have been coded according to the International Classification of Diseases Revision 8 (ICD-8) from 1977 to 1993 and International Classification of Diseases Revision 10 (ICD-10) since 1994 by discharging physicians at the time of hospital discharge.²⁵ The registry automatically checks for possible errors and discrepancies and sends them back to hospital systems for correction.^22,25 The DNPR gives valuable and specific information on diagnoses, treatments, and examinations of patients.^22,25 Previous studies have shown that DNPR is valid for reporting comorbidities,²⁶ previous orthopedic surgery,²⁵ diagnostic procedures,²⁷ prescription-based medications²³ and in-hospital medical treatments.²⁵

The Danish National Prescription Registry contains data on all prescription drugs since 1995. Type of drug, date of dispensing, and quantity are included in the data. The Anatomic Therapeutic Chemical (ATC) index is used to categorize the drugs. Since the data on Danish National Prescription Registry constitute a key component of selling prescription drugs in pharmacies, inter/intraobserver information bias are very unlikely to occur in prescription registry.²³

The Danish Civil Registration System allows linkage between registries, extraction of population controls, and tracking of Danish citizens. High data quality in the Danish Civil Registration System is ensured by law and several processes, such as ongoing validation and error correction of the recorded information.²⁴

Data Analysis

Two researchers extracted and uploaded the data from the GLA:D registry to a secure server environment at The Danish Health Data Authority. Afterwards GLA:D data was linked to the national Danish registries on the server via the CPR number. The rules of construction of the CPR numbers were implemented in the GLA:D data collection and only valid CPR numbers were allowed to be entered. The combined dataset on the server was analyzed by one researcher. For comparability of the GLA:D and registry-based data, GLA:D registry data were translated into ATC Classification System codes for prescription drugs, International Statistical Classification of Diseases and Related Health Problems (ICD-10) codes for comorbidities and web-based Health Care Classification (SKS) system codes for non- and surgical hospital procedures. Algorithms that combine ICD codes with prescription codes increase the probability of identifying comorbidities in DNPR,²⁵ therefore, we also used ATC codes in addition to ICD codes to increase the probability of identifying registry-based comorbidities. Details of variables and adaptation of the data from GLA:D registry to fit the available data from Danish registries for comparability are presented in Appendix 1. ATC codes were extracted from the National Prescription Registry if the patient had a prescribed medication purchase 3 months prior to index date. SKS codes before index date and, SKS codes between the index date and the date of reporting the 12 months follow-up data were extracted from DNPR for each patient. Radiographs prior to index date were extracted from DNPR for the most affected joint. If the side of radiograph was not available in the DNPR, the patient was excluded from the analysis. Comorbidities were defined as having a relevant diagnosis within 5 years prior to index date or redeeming a prescription medication within 1 year prior to index date. Missing datapoints in GLA:D were also set to be missing in the corresponding registry variable. Details of the ICD-10, SKS and ATC codes are presented in Appendix 2. Then, we evaluated the agreement between the GLA:D data and registry-based data.

We calculated the standard statistics of Cohen’s kappa and percentage agreement (ie how many yes and no in GLA:D that correspond to yes and no in the national registries, respectively), with 95% confidence intervals where appropriate. Kappa values <0 interpreted as no agreement, 0.00–0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement.²⁸ All data analysis was conducted using Stata 16.1.²⁹

Results

Of 38.745 participants, 27,669 (71%) were female. The mean age of the participants was 65 (± 9.88) years. The characteristics of the participants are described in Table 1.

Table 1 Characteristics of the Study Participants

There was a perfect agreement between self-report and registry-based data for baseline joint replacement surgery, with a kappa value of 0.95 (95% CI= 0.94–0.96) and agreement of 99.54% (95% CI= 99.46–99.61) for knee and a kappa value of 0.95 (95% CI= 0.94–0.96) and agreement of 99.48% (95% CI= 99.40–99.55) for hip (Table 2). There was a perfect agreement between self-report and registry-based data for baseline to 12-month follow-up joint replacement surgery, with a kappa value of 0.91 (95% CI= 0.89–0.92) and agreement of 99.15% (95% CI= 98.99–99.28) for knee and a kappa value of 0.94 (95% CI= 0.92–0.96) and agreement of 99.45% (95% CI= 99.32–99.56) for hip (Table 2).

Table 2 Agreement Between Self-Report and Registry-Based Data for Baseline and Baseline to 12-Month Follow-Up Replacement Surgery

The agreement between self-report and registry-based data for the type of prior surgery of the knee and hip joints collected by the physiotherapist is presented in Table 3. There was a moderate agreement between self-report and registry-based data for previous knee surgery (kappa: 0.58, agreement: 84.99%) and a substantial agreement for previous hip surgery (kappa: 0.73, agreement: 97.05%). For previous knee surgery subgroups, kappa values ranged between 0.05 and 0.95 and agreement ranged between 96.04% and 99.94%. The agreement was slight to almost perfect with the lowest agreement for collateral ligament surgery and highest agreement for joint replacement. For previous hip surgery subgroups, kappa values ranged between 0.17 and 0.94 and agreement ranged between 99.50% and 99.94%. The agreement was slight to almost perfect with the lowest agreement for other surgeries and the highest agreement for joint replacement.

Table 3 Agreement Between Self-Report Collected by Therapist and Registry-Based Data for Previous Surgery of the Knee and Hip Joints

The agreement between self-report and registry-based data for different types of pain medication is presented in Table 4. Kappa values ranged between 0.01 and 0.53 and agreement ranged between 65.39% and 99.90%. The agreement was slight to moderate with the lowest agreement for topical NSAID and highest agreement for opioids. There was a moderate agreement (kappa: 0.41, agreement: 81.59%) for knee radiographs and a slight agreement (kappa: 0.20, agreement: 64.79%) for hip radiographs (Table 4).

Table 4 Agreement Between Self-Report and Registry-Based Data for Different Types of Pain Medication and Radiograph Data

The agreement between self-report and registry-based data for comorbidities is presented in Table 5. Kappa values ranged between 0.14 (95% CI= 0.13–0.15) to 0.90 (95% CI= 0.89–0.91) and agreement ranged between 78.07% (95% CI= 77.61–78.52) to 98.91% (95% CI= 98.49–99.24). The agreement was slight to almost perfect with the lowest agreement for anemia or other blood diseases and the highest agreement for diabetes. The details of the answer distribution in GLA:D data and registry-based data for each variable can be found in Appendix 3.

Table 5 Agreement Between Self-Report and Registry-Based Data for Comorbidities

Discussion

In this study, we investigated the agreement between self-reported information from the GLA:D registry and registry-based data from national registries. We found a perfect agreement for replacement surgery of knee and hip, moderate agreement for previous knee surgery, and substantial agreement for previous hip surgery between the GLA:D and registry-based data. When categorizing surgeries into subgroups, the agreement level ranged from slight to almost perfect for different types of surgery with a higher agreement when collapsing into less detailed categories of surgery. For medication subgroups, agreement ranged from slight to moderate with the highest agreement on opioids. The agreement also ranged from slight to almost perfect for comorbidities with the highest agreement on diabetes, while the agreement for having had a radiograph was moderate for knee radiographs and only slight for hip radiographs.

Surgery and Imaging

Surgical procedures have been coded in DNPR according to the Nordic Medico-Statistical Committee (NOMESCO) system since 1996.³⁰ Previous studies have shown that data on surgical procedures in DNPR is valid.^31–33 The results of this study indicate that there is almost perfect agreement between GLA:D and registry-based data on the knee and hip replacement surgery at baseline and follow-up. These findings are consistent with previous studies that found substantial to almost perfect agreement between self-report and administrative data.^10,34

For different types of surgery, the agreement varied substantially between types but increased when collapsed into less detailed categories of surgery. Besides potential misclassifications in DNPR, this indicates difficulty in discriminating between detailed surgery types for the patients and suggests that less detailed categories should be used when asking patients to self-report prior surgery. Also, the high agreement on joint replacement compared to other types of surgery might be due to the fact that it is easier to recall major operation that is mostly performed later in life compared to other surgery types like arthroscopy.

Since 2002, it has been mandatory for all Danish hospitals to report radiological procedures to the DNPR.³⁵ In our study, there was a moderate agreement for the knee and slight agreement for the hip between GLA:D and registry-based data on radiographs with more radiographs reported in GLA:D. Previous studies evaluating the completeness of radiograph registrations in DNPR showed low completeness (73.1%),³⁵ which question using the DNPR as the gold standard, and may partly explain the lower number of radiographs reported in the national registry compared to self-report in our study. Another potential reason for the overreport of radiograph data in GLA:D compared to registry-based data might be that the radiographs taken in primary care and radiographs taken before 2002 are not recorded in DNPR and also, patients might not know the difference between types of imaging (eg MRI and ultrasound). The reason for the different agreement levels for knee and hip might be that bilateral problems are common, and patients could not accurately recall the joint of which they had a radiograph.

Pain Medication

The Danish National Prescription Registry contains information by CPR number on all drugs prescribed and dispensed in Danish outpatient pharmacies.³⁶ We found an overreporting for paracetamol and NSAIDs in the GLA:D data compared to the registry-based data, contributing to a fair agreement. This confirms previous findings by Grimaldi‐Bensouda et al, who investigated the agreement between physician-reported and patient-reported drugs used for musculoskeletal disorders and showed fair agreement for NSAIDs.³⁷ This might be due to the over-the-counter availability of paracetamol and NSAIDs, which are not registered in the national registry.³⁸ Also, since the patients in GLA:D were asked about pain medication use, but we retrieved dispensations from the national prescription registry, patients could have had medications in their home that were dispensed at an earlier time point or they could have dispensed medications that they did not use which might lead to a disagreement between the two sources.

Similar to our results, a previous study on the Danish population also showed a moderate agreement between self-reported and registry-based data for opioid use.³⁹ Opioids are normally used in OA treatment when needed and not on a day-to-day basis, which might be part of the explanation for the underreporting of opioids in GLA:D, as it can be difficult to report medications that are not used regularly.⁴⁰

There was a fair agreement for hyaluronic acid injection and corticosteroid injection. Since another limitation of the registry-based data is sensitivity towards prescriptions issued by private-practicing specialists,³⁸ this might explain the rather low agreement.

Comorbidities

We found almost perfect agreement for Diabetes, but the agreement was moderate when this category was divided into Diabetes type 1 and Diabetes type 2 from 26.08.2018 and onwards. Some patients might not be able to discriminate between the two types of diabetes and therefore the agreement was lower when asked about diabetes type rather than diabetes in general. Also, this might be due to the ICD code definition, as the category of Diabetes is covered by codes E10 to E14, while Diabetes type 1 is only defined by E10 Diabetes type 2 by E11. Therefore, codes E12 to E14 are ignored when divided into Diabetes type 1 and Diabetes type 2. There was also a substantial agreement for high blood pressure, high cholesterol and moderate agreement for heart disease. Similar to these results, previous studies have also demonstrated substantial agreement for diabetes between self-report and medical record data and also a moderate to substantial agreement for cardiovascular problems.^41,42

There was a fair agreement for cancer. The rather low agreement on cancer might be partly explained by the fact that patients in GLA:D are asked about current conditions, while the DNPR would also include cancer that the patients have had within past 5 years, ie, patients in GLA:D do not report the previous cancer cases for which they have been cured. We also found a fair agreement for rheumatoid arthritis (RA), explained by a higher prevalence according to the self-reported information. A potential reason for the higher numbers in GLA:D might be due to the low validity of the DNPR for detecting RA, which has been shown in previous studies,⁴³ but also due to misreporting in GLA:D, as the number of cases with rheumatoid arthritis dropped significantly after the label added that rheumatoid arthritis is not the same as OA on 10.07.2015.

For other conditions, the agreement varied from slight to moderate. There was a moderate agreement for chronic lung disease and depression, a fair agreement for kidney or liver disease and a slight agreement for ulcers or other stomach disease, neurological disease and anemia or other blood diseases between GLA:D and registry-based data.

Similar to these results, a study by Koller et al investigated the agreement between self-report and medical record data on comorbidities among 3821 residents of three regions from Alaska, United States of America with a population survey and showed slight to substantial agreement. The agreement was highest for diabetes and cancer, on the contrary, the agreement was lowest for chronic obstructive pulmonary disease (COPD), kidney, and heart disease. Also, the agreement levels differed by age, as the kappa values were lower in adults 65 years or older.⁴⁴ Other studies also found low agreement between self-report and administrative data on the number of comorbidities in adults older than 45 years⁴⁵ and elderly patients at emergency visits.⁴⁶ The mean age of the population in this study was 65 years, and therefore agreement between self-reported and registry-based data on comorbidities might also be affected by age. Consequently, this study demonstrated a slight to almost perfect agreement for different comorbidities, which is consistent with the previous studies.

We recommend that future studies should focus on factors affecting the validity of self-reported information obtained from patients with knee or hip OA.

Strengths and Limitations

The main strength of this study is the large sample size and coverage of different aspects of medical history. This study also has some limitations. First, even though DNPR has been shown to be valid in previous studies, it also has some limitations as discussed above. Secondly, missing data in the national registries might be a limitation of this study.²⁵ Thirdly, we investigated the agreement in the Danish population and therefore these results might not apply to different cultures or countries. Fourthly, since we only had self-reported information on x-ray findings in GLA:D, we were not able to evaluate the radiographic severity of the patient’s OA or determine whether they in fact had an x-ray or another type of imaging of their joint. Lastly, GLA:D^® information was collected by 923 physiotherapists, and this could contribute to variations in data collection. On the other hand, since all physiotherapists were specifically trained in the GLA:D^® data collection and treatment procedures, this could be considered a strength as it is real-world clinical data resembling everyday clinical practice.

Conclusion

As the most common types of pain medication used by patients with OA can be bought over-the-counter and as most OA patients are treated in primary care, which is often not covered by national registries, self-report of pain medication use, and comorbidities is preferred but cannot be sufficiently validated against registry-based data. Future studies collecting self-reported information on joint surgery and pain medication from patients with OA should use a less detailed categorization to improve accuracy.