High Completeness and Accurate Reporting of Key Variables Make Data from the Danish Shoulder Arthroplasty Registry a Valuable Source of Information

Introduction

As any other medical implant, shoulder arthroplasties can be released to the market without any documentation of clinical efficacy or adverse effects. Randomized clinical trials are laborious, expensive and time-consuming and are rarely conducted to test new implants against the gold standard. In this perspective, large arthroplasty registries like the Danish Shoulder Arthroplasty Registry (DSR) play an important role in implant approval and post-marketing surveillance.^1,2

Shoulder arthroplasty is used for various indications, most commonly glenohumeral osteoarthritis, proximal humeral fracture or rotator cuff arthropathy. The use of shoulder arthroplasty in Denmark has been monitored by the DSR since 2004. Revision, defined as removal, exchange or addition of any component, and the Western Ontario osteoarthritis of the Shoulder (WOOS) index at one year are used as outcomes.³

The purpose of the registry is to improve patient-reported outcome and arthroplasty longevity on a national level, to ensure consistent indication and use of arthroplasty type across the country, and to ensure that no department or implant perform significantly worse than expected. Data from the registry are sent to national and regional healthcare authorities each month and used for annual reports where the results are interpreted by the steering committee. The results are presented and discussed at the annual meeting of the Danish Orthopaedic Society. Data has also been used for several scientific papers exploring hypotheses about risk factors for revision, and risk factors for an inferior patient-reported outcome.^4–6

To be useful for any type of publication, registry data depend on good data quality. Low completeness of reporting may compromise the validity, especially if there are significant differences between registered and non-registered arthroplasties. Incorrect reporting of variables may lead to false conclusions and recommendations.⁷

The aim was to validate data from the DSR by (1) assessing the registration completeness, (2) comparing key variables with information from medical records, (3) assessing the number and proportion of missing data for key variables.

Methods

The Danish Shoulder Arthroplasty Registry and Study Population

Denmark provides tax-paid healthcare services for all Danish Citizens (~5.6 million persons), allowing free access to general practitioners and hospital services including shoulder arthroplasty surgery. Healthcare services is administrated by five Danish counties. Different electronic medical record systems were implemented in each county between 2006 and 2008.

The DSR was founded in 2004 by the Danish Society for Shoulder and Elbow Surgery. It is financed by the Danish healthcare authorities and is independent of commercial interests. The DSR shares a common annual budget of 160.000 Euros with the Danish Hip Arthroplasty Registry, the Danish Knee Arthroplasty Registry and the Danish Knee Ligament Reconstruction Registry.

Reporting to the registry requires no patient consent and has been mandatory for all public hospitals and private clinics in Denmark since 2006. The completeness of reporting in 2004–2005 was low and these two years are regarded as a trial period. Detailed data on all arthroplasties are reported by the surgeon at the time of surgery using a standardized web-based reporting system.³

Validation of Completeness

The completeness is calculated by comparing the number of arthroplasties collected by the DSR with the number of arthroplasties collected by the DSR and Danish National Patient Register (DNPR) together. DNPR is an administrative database used by the Danish healthcare authorities to monitor all hospitalizations and treatments including shoulder arthroplasty surgery.^8,9 Data from the DNPR include date of surgery and discharge and surgical code using the Nordic Medico-Statistical Committee (NOMESCO) system.¹⁰ The following NOMESCO code is regarded as a primary arthroplasty: KNBB. We assessed the completeness for the entire study period and for each year.

All Danish residents are assigned a unique civil registration (CPR) number at birth or immigration. The civil registration system uses the CPR number to collect information about residence, vital status and emigration. Information is updated every day.^11,12 The CPR number is used to accurately link data from the DSR with data from the DNPR. To ensure high completeness of reporting, data from the DSR are linked to the DNPR every 3 months. In case of discrepancy between DSR and DNPR, a reminder is sent to the orthopedic department which include the CPR number, the surgical code and the dates of surgery and discharge.

Validation of Key Variables

The validity of key variables within the DSR was assessed by auditing medical records from a random sample of registrations in the DSR from 2006 to 2015. Based on previous and similar validations of Danish orthopedic registries^13,14 we assumed a positive predictive value (PPV) of 80% for the most inexact variables, and to estimate the PPV with a precision of 5%, a sample size of 240 primary arthroplasties were required. To account for irretrievable medical records, 300 primary arthroplasties were included in this part of the study. The sample was randomly selected by computation from four strata based on year of surgery (2006–2008, 2009–2011, 2012–2013 and 2014–2015) to depict the composition of registrations in the DSR.

The key variables were validated using data from medical record and postoperative radiographs of the glenohumeral joint as gold standard. Data from medical records were retrieved using the CPR number and entered into a database using the REDCap (Research Electronic Data Capture) software. A study protocol, which explicitly defined variables and categories, was prepared prior to the study. The protocol included pictures and radiographs of all brands and arthroplasty types enabling the observers to identify the brand and arthroplasty type based on postoperative radiographs alone. Three shoulder surgeons served as observers. Disagreements between the three observers regarding classification were examined for 12 cases and subsequently solved in consensus before data collection was initiated. The observers were blinded to data in the DSR. All key variables had to be assessed before the registration could be completed. The key variables included date of surgery, side, indication, arthroplasty type and other variables important to the annual report and peer-review publications (Table 1).

Table 1 Validation of Key Variables in the DSR

Key Variable Completeness

The completeness of the key variables was evaluated for DSR between 2006–2015. The missing values were evaluated for year of surgery, age, gender and hospital to depict if any systematic linkage could be observed, eg, if the majority of missing values within a key variable were associated with a specific time period. Observations with missing values were analyzed to depict the number of missing values within an observation and any traits between the missing values, eg, if missing values in glenoid anchoring predisposes for missing values in glenoid fixation or design. In combination, these analyses were used to assess the likelihood for the key variables being missing at random (MAR) or missing not at random (MNAR).

Statistics

The completeness of reporting was calculated as:

A 2 x 2 table was used to calculate the PPV for each key variable. The PPV was calculated as the number of registrations in the DSR with their key variable confirmed in the medical records divided by the number of positive registrations in that key variable in the DSR.

To estimate 95% confidence interval (CI) we relied on the normal approximation of the binomial distribution. Baseline values from the key variables were presented by their distribution, mean and standard deviation (SD) or median and inter quantile range (IQR), as suited. Analyses were done using the SPSS version 22.0 (IBM SPSS Statistics for Windows, Version 22.0. IBM Corp, Armonk, NY, USA) and STATA 16 (StataCorp LLC, College Station, Texas, USA).

Results

Baseline Characteristics

A total of 8976 primary shoulder arthroplasties were registered in the DSR between January 1. 2006 and December 31. 2015. The number of primary arthroplasties increased from 2006 to 2015, Table 2. Mean age was 70 years (SD: 11 years) and 68% were females. Seven thousand nine hundred and thirty-four (88%) arthroplasties were unilateral and 1042 (12%) were bilateral.

Table 2 Completeness of Reporting

Registry Completeness

In 2006, the completeness of reporting was 81% (95% CI: 78–84%). From 2007 to 2015 the completeness ranged from 92% (95% CI: 91–94%) to 98% (95% CI: 97–99%). The overall completeness in the study period was 94% (95% CI: 94–94%) (Table 2).

Validation of Key Variables

Medical records could not be retrieved for 24 (8%) out of 300 arthroplasties. Fifteen missing records were from patients who died before 2016. Their medical records were not transferred to a renewed electronic medical record system in 2016. Six missing records were from patients who were operated before the introduction of electronic medical records, and whose original paper records were irretrievable. Three missing records were from patients operated at private hospitals which had either closed or were unable to deliver data.

The PPV for the three major indications: osteoarthritis, fracture and rotator cuff arthropathy was 92% (95% CI: 85–96%), 97% (95% CI: 92–99%), and 94% (95% CI: 86–98%), respectively. The PPV was lower for arthritis, avascular humeral head necrosis and “other indications”. Together these indications accounted for 15% of the sample (Table 1). Twelve patients were registered with avascular humeral head necrosis in the DSR, but only 7 of these were confirmed in the medical records. The remaining five indications were recorded as fractures (n=3) and osteoarthritis (n=2). Eighteen patients were registered with arthritis in the DSR, but only 13 of these were confirmed by the medical records. The remaining indications were registered as osteoarthritis (n=5) in the medical records. Nine patients were registered with other indication in the DSR, but only 3 of these were confirmed by the medical records. The remaining 6 were fractures (n=3), cuff arthropathy (n=2) and osteoarthritis (n=1)

The PPV for arthroplasty type varied. It was 93% (95% CI: 80–98%) for reverse shoulder arthroplasty and 93% for resurfacing arthroplasty (95% CI: 66–100%), but only 77% (95% CI: 62–88%) and 83% (95% CI: 76–88%) for total shoulder arthroplasty and hemiarthroplasty (Table 1). Forty-seven total shoulder arthroplasties were registered in the DSR, but only 36 of these were confirmed by the medical records. The remaining 11 arthroplasties were noted as reverse shoulder arthroplasty (n=8), resurfacing arthroplasty (n=1) and hemiarthroplasty (n=2) in the medical records. The moderate PPV for hemiarthroplasty was mainly related to patients with osteoarthritis. Thus, 43 hemiarthroplasties for osteoarthritis were registered in the DSR, but only 18 of these were confirmed by the medical records. The remaining arthroplasties were noted as resurfacing arthroplasty (n=25).

Key Variable Completeness

The proportion of missing data in DSR was less than 1% for age, gender, previous surgery, indication and arthroplasty type (Table 3). Patients with missing value were evenly distributed throughout the study period, and comparable to the complete observations regarding age and sex. Eleven (0.1%) observations had missing arthroplasty brand, and 21 (0.2%) observations had missing arthroplasty type. Fixation technique of the humeral component was only included in the dataset from 2006–2009. One thousand two hundred and one (42%) observations were missing in this period. Glenoid anchor design, material and fixation were missing in 518 (17%), 729 (24%) and 602 (20%) of the 3015 observations with glenoid component. Observations with missing values in glenoid characteristics were evenly distributed throughout the study period, and comparable to the complete observations regarding age and sex. Two hundred and eleven (7%) of the missing observations had missing values in one of three glenoid characteristics, 241 (8%) had missing in two of three and 362 (12%) had missing observations in all three glenoid characteristics.

Table 3 Missing Data in Key Variables Within the Danish Shoulder Arthroplasty Registry

Discussion

Validation of Completeness

The completeness of reporting was similar to the completeness reported by the Danish Hip arthroplasty registry,¹⁴ the Danish Knee Ligament Reconstruction Registry¹³ and to the completeness reported in annual reports from other national shoulder arthroplasty registries.^15,16 It is important to bear in mind that the results are based on different administrative databases as gold standard. The completeness may, therefore, not be directly comparable with the completeness in other national shoulder arthroplasty registries.

The reason for the high completeness is speculative. It might be related to an easily accessible electronic reporting form which can be completed within few minutes, the use of reminders to orthopedic departments in case of missing reporting, and the fact that patient consent is not required for data reporting. Nevertheless, the completeness is not 100% despite mandatory reporting. Missing reporting may be related to economic or surgeon-related resources, limited knowledge about the registry and data reporting or technical difficulties. Orthopedic departments can, in theory, lose their license to perform shoulder arthroplasty surgery in case of missing reporting. This reprisal has, however, never been used by the healthcare authorities.

Reporting of shoulder arthroplasty to DNRP has never been validated, and inconsistent, incorrect or missing reporting could influence the results. By using a gold standard defined as arthroplasties collected by either DSR or DNRP the risk of bias is reduced.

Validation of Key Variables

We regard the PPV for the key variables as high, even though there is no consensus on an arbitrary limit of an adequate PPV. There is, to our knowledge, no reporting on the PPV for key variables from other national shoulder arthroplasty registries, which we can use for comparison.

The lowest PPV for diagnosis was found for humeral head necrosis. This is probably related to the reporting system where more than one diagnosis could be reported. Humeral head necrosis was often reported together with other diagnoses such as fracture or osteoarthritis. Clear definitions and the use of a hierarchy of diagnosis where surgeons only report the highest-ranking diagnosis would likely have increased the PPV for diagnosis.

The lowest PPV for arthroplasty type was found for total shoulder arthroplasty. This is probably related to mutually nonexclusive reporting possibilities and inadequate definitions of arthroplasty types. Thus, the reverse shoulder arthroplasty may also be regarded as a total shoulder arthroplasty and was reported as such in 11 cases (23%). We observed a similar problem with resurfacing arthroplasty being reported as hemiarthroplasty, but, in this case, the PPV for hemiarthroplasty was higher. This was related to a high number of hemiarthroplasties for fracture. For patients with osteoarthritis, 25 (58%) resurfacing arthroplasties were reported as hemiarthroplasty.

The non-exclusive reporting and inadequate definitions of arthroplasty types have now been changed. Today, arthroplasty type is not defined and reported by the surgeon, but instead extracted from the reporting of the individual arthroplasty components that have been implanted. We recommend this approach or even better, bar-code registration, which would reduce data registration errors and workload for the surgeon. If this is not possible it is important to use an exclusive reporting method with clear definition of the different arthroplasty types.

Key Variable Completeness

There were no missing data for age and gender as this information was given by the CPR number. There were no missing data for side and indication for surgery and less than 1% of the observations had missing data for previous surgery, brand and arthroplasty type.

As the amount of missing data were low and observations with missing variables did not differ from complete observations, we recommend that missing values are considered as missing at random in future studies and annual reports.

Information about fixation technique of the humeral component was missing in 42% arthroplasties. Glenoid anchor design (pegs, keeled, screws), material (all-polyethylene, metal-backed) and fixation (cemented, uncemented) were missing in 17%, 24% and 20% of the arthroplasties with a glenoid component. The reason for this is unknown. Results which include data on fixation technique of the humeral component and glenoid component characteristics should be interpreted carefully.

For all variables, making fields obligatory in the electronic form would be a way to eliminate missing values.

Strength and Limitations

The strength of the present study was the possibility of collecting data from medical records for more than 90% of the patients in our computer-generated nationwide random sample. The systematic collection of data from medical records and the use of postoperative radiographs limits the risk of information bias. Furthermore, the reviewers of the medical records were blinded to data in the DSR when entering cases into REDCap. Finally, the study population is unselected and represent all hospitals and private clinics in Denmark. Thus, the results do not reflect the reporting of a few centers with a special interest in registry data.

The limitations of the study are mainly related to the use of medical records as gold standard in the validation of key variables. We were unable to assess 8% of the medical records and in some cases, we were unable to retrieve information for all key variables. The results can be biased if missing data have a higher or lower proportion of incorrect reporting. Furthermore, incorrect information in medical records may have influenced the results. Finally, for some key variables (eg, shoulder arthroplasty type) subgroups were small resulting in statistical imprecision with wide confidence intervals.

Use of DSR Data and Methodological Considerations

This study only reports the accuracy of data reported by the surgeon at the time of the primary operation. There is no information about the completeness of WOOS or the completeness and accuracy of reporting for revision arthroplasties which are crucial to the use and interpretation of DSR data.

The completeness of WOOS has been reported in a previous publication where the authors found that 65% of patients returned the WOOS questionnaire. One single postal reminder increased the responds rate to 80%, and a telephone follow-up further increased the responds rate to 82%. There was no statistically significant difference in WOOS between responders to the original questionnaire and the responders to the postal reminder or the telephone follow-up. Consistent non-responders (18% of the patients) showed no statistically significant differences in gender or diagnosis compared with the pooled group of all responders, but they were generally younger. The authors concluded that non-responders did appear to bias the results after shoulder arthroplasty.¹⁷

Reporting of completeness and validation of key variables for revision arthroplasty were not within the scope of this study but it should be a subject for future research. If data from the registry are used to study rare events like risk of revision because of periprosthetic joint infection or dislocation after reverse shoulder arthroplasty it is important to know not only the completeness of reporting but also the accuracy of the reported reasons for revision.

Previous annual reports and peer review publications from DSR have included bilateral replacements as they were independent. From a statistical point of view, the inclusion of bilateral observations violates the assumption of independent observations and thus it would be more appropriate to only include the first arthroplasty. When investigating implant survival in large arthroplasty studies, the potential bias from bilateral observations seems neglectable.¹⁸ However, the consequences of ignoring the assumption of independency when using WOOS as outcome are unknown, and the dependency issue should be kept in mind when results are being interpreted.

The PPV for diagnosis was generally good. The reporting of osteoarthritis, fracture and cuff tear arthropathy can be regarded as accurate. However, this study raises concerns regarding the reporting of humeral head necrosis. The low PPV could lead to false conclusions especially if there is unequal distribution of misclassifications between subgroups.

The PPV for hemiarthroplasty and total shoulder arthroplasty was lower than for other arthroplasty types. Previous publications from DSR have used the arthroplasty brand to manually adjust the arthroplasty type. Thus, for frequently used brands like the Copeland and the Delta Xtend the arthroplasty type can be extrapolated from the arthroplasty brand and subsequently adjusted. Today, information about arthroplasty type is extracted from the reporting of arthroplasty components that have been implanted. The lower PPV for some arthroplasty types is expected to have little practical consequences when data from DSR are analyzed and interpreted.

The PPV for type of previous surgery and technical aspects of the arthroplasty procedure was often based on few cases and with wide 95% CIs. We cannot make safe recommendations regarding the future use of these variables, but the low PPV for some variables is worth considering when future studies are planned.

Conclusion

The study showed that data from the DSR are sufficiently valid to be used for research and quality monitoring. Lower PPV’s for total shoulder arthroplasty and hemiarthroplasty are possibly related to inadequate definitions and mutually nonexclusive items in the reporting form. We recommend that surgeons report the individual type of arthroplasty component that have been used, or even better bar-code registration, to reduce data registration errors. Regular validation is necessary since the data reported to the registry continuously evolve because of changes in clinical practice.