Disparate Impacts of Two Public Reporting Initiatives on Clinical and Perceived Quality in Healthcare

Introduction

The healthcare market has been largely directed by a quality-blind payment system that promotes volume-based services without consideration for the quality of care,¹ and transparency is increasingly viewed as a prerequisite for value-based health care that invites quality in the assessment of achieved value. The executive order on “Improving Price and Quality Transparency in American Healthcare to Put Patients First” by the Trump Administration explicitly noted the significance of transparent information in health care. As the engagement of consumers and patient-centeredness have become central to health policy in the US, public reporting has become a crucial building block to assist patients and their families in making informed choices on health plans, health providers, clinical examinations, or alternative treatments.²

At present, the ability of transparency initiatives to enhance quality of care remains obscure, if not rejected. Proponents have argued that allowing performance data to be publicly available has often been proposed as a tool for achieving better quality of care by means of providing transparency and accountability from their health care providers.^3–5 Not only will it guide purchasers when selecting insurance plans, but it will also motivate providers to outperform in order to protect their reputations and the demand for their services.⁶ It is also expected that hospitals would commit more readily to patient-centered and safe services as health care consumers increasingly expect fair value for their money.

Despite the government’s bold steps toward transparency, however, the Centers for Medicare and Medicaid Services (CMS) has been experiencing resistance from the healthcare industry. Skeptics believe that public reporting initiatives may introduce the potential problem of risk aversion, which implies that hospitals may strategically avoid severely ill patients who would harm their reputation under higher transparency.^7,8 There was a case where the expected mortality rate of coronary artery bypass graft (CABG) notably increased after disclosure of performance information because high-risk patients tended to choose outperforming hospitals when performance data were released.⁹ The American Hospital Association (AHA) contended that disclosing privately negotiated rates will not help patients understand but create further confusion about their actual payment for treatment.¹⁰ Moreover, the Blue Cross Blue Shield Association (BCBSA) claimed that compliance with the transparency requirements would cost $13.63 million, which is 26 times higher than the government had estimated.¹¹

Given the widespread belief in the virtue of transparency and the resistance to it at the same time, health care providers’ performance on quality needs to be systematically evaluated in pursuit of desirable transparent environments in the health care system. It is essential to identify how transparency initiatives influence quality of care. Furthermore, understanding how the transparency mechanism works in the healthcare market will allow health policymakers to make evidence-based decisions that build a better functioning healthcare system in the US. In so doing, public disclosure of hospital performance information will eventually be able to foster a spirit of openness and trustworthiness in healthcare which has long exhibited serious asymmetries of information.^12,13 In the context of hospital management, performance assessment enables hospital managers to be aware of their own status in the market. Knowing how they have behaved will result in organizational learning and advancement.

The objective of this study is to offer solid scientific evidence on the impact of transparency policy. The present study has two unique contributions to the literature. It investigates policy impact along with hospital market competition. Market competition is central because it offers a fundamental rationale that transparency policies theoretically rely upon. Health care reforms in the US have historically been geared toward market-oriented remedies ever since the Reagan presidency in the 1980s.¹⁴ More importantly, this study incorporates two disparate types of quality measures: clinical quality and perceived quality. Clinical quality involves clinical operation-based quality measures, often captured by healthcare-associated infections (HAIs) and 30-day readmission rates. The perceived quality is measured based upon patient experience, represented by patient satisfaction. In healthcare, patients are key participants in producing health services along with the providers.¹⁵ Therefore, the health service quality of hospitals must be assessed based on clinical outcomes as well as patients’ experience. These are all significant quality metrics that healthcare providers must squarely consider in their quality improvement process.

Literature Review and Hypotheses

A number of states currently have public reporting programs in place. Some of these efforts are publicly funded, while others are operated by nonprofit organizations, community collaboratives, or large purchasers of health care to achieve better quality of care.^6,16 An increasing number of government agencies, private companies, nonprofit organizations, and even health care organizations offer certain types of price, quality, and satisfaction data on health care providers.¹⁷ The transparency initiative of interest in this study is all-payer claims databases (APCDs). The APCDs are state-level archives that aggregate statewide health care data from various payers, and they are expected to reduce health care prices through competition fostered by informing consumers. The reason why this research incorporates the APCDs initiative among others is that it comprehensively collects a variety of health care payers, including private health insurers, physician and hospital files, prescription drug plans and state employee health benefit programs, not to mention Medicaid. A growing number of states are implementing the All-Payer Claims Databases (APCDs) policy.

The extant literature suggests three pathways through which disseminations of performance data can contribute to improved quality of care. First, the selection pathway assumes that higher quality of care would be achieved as hospitals make efforts to perform better in order to maintain, and not lose, their market share and attract patients.¹⁸ Second, the reputation pathway, based on the concern of health care providers about their public image, can be impaired by underperformance on quality. Thus, health care providers are expected to seek quality improvements.^18,19 The last pathway is related to providers’ professional motivation.²⁰

Identification of the quality chasm itself may drive health care organizations to actualize higher service quality.

H1. A transparent environment is positively associated with the clinical quality of a hospital.

Public disclosure of cost and quality information is designed to address the imbalance in information in the market. Uncertainty and information asymmetry will be reduced by enforcing transparency policies as the new rules of the game in the health care market.²¹ Accordingly, it is expected that the efforts of hospitals to survive the market would stand out more in competitive markets than in concentrated markets.²² Furthermore, health care providers are expected to conform to the new institutional arrangement in order to survive in the market, as elements of transparency are added to the market structure.^22,23 If the competition mechanism starts to work by allowing richer health care information to flow into the market, it is reasonable to expect hospitals to respond in turn, even without consumer response. When health care administrators and managers receive feedback from the publicly available quality measures and factor them into their organizational strategies, those hospitals under the clearer rule of transparency are likely to perform better.²⁴ The new institutional context will eliminate poor quality performers and reward good-quality performers by bringing a greater volume of patients to high performing hospitals.²⁵ Regarding the relationship between utilization and public reporting, the extant literature cited consistent evidence of health care consumers’ selection of better rated hospitals.^26,27 Based on these reasonings, this study hypothesizes the following:

H3 Market Competition is positively associated with the clinical quality of hospital.

Materials and Methods

Construction of the Variables

Dependent Variables

The dependent variables are the factor loading scores of the three proxies: quality of care using multiple measures of HAIs, 30-day readmission rate, and patient satisfaction. Factor analyses were conducted using principal-components factoring. Details of factor loadings are reported in Table 1. The component of HAIs was measured by six variables: (1) Central Line Associated Bloodstream Infection (CLABSI); (2) Catheter Associated Urinary Tract Infections (CAUTI); (3) Surgical Site Infection (SSI) during Colon Surgery; (4) SSI during Abdominal Hysterectomy; (5) Methicillin-Resistant Staphylococcus aureus Bacteremia (MRSA); and (6) Clostridium difficile Infection (C.Diff). The hospital-level results for HAIs measure the estimate of excess readmissions to any acute care hospital within 30 days of discharge from a hospitalization. The analysis retained one factor, which accounts for over 73.6% of the data variation (Cronbach’s = 0.736).

Table 1 Data Descriptions

The ratio of excess readmissions is calculated by dividing a hospital’s number of “predicted” 30-day readmissions by the number that would be “expected,” based on the average hospital with similar patients. There are six medical conditions included for the 30-day readmission rate component measures: heart attack (AMI), coronary artery bypass graft (CABG), chronic obstructive pulmonary disease (COPD), heart failure (HF), hip and knee replacement, and pneumonia (PN). This factor explains more than 78.8% of the data variation (Cronbach’s = 0.788), which shows a high internal consistency.

The component of patient satisfaction was measured by nine variables: (1) how patients were satisfied with doctor communication; (2) how patients were satisfied with nurse communication; (3) responsiveness of hospital staff; (4) communication about medicines; (5) cleanliness of hospital environment; (6) quietness of hospital environment; (7) discharge information; (8) care transition; and (9) willingness to recommend this hospital. Although some healthcare associations call for improved data measurement and presentation,²⁸ the star ratings are the most comprehensive and comparable data on health care quality publicly available to date. The hospital rating ranges from 1 to 5 stars and shows how each hospital performed, the higher the better. This factor explains more than 97.8% of the data variation (Cronbach’s = 0.978), which presents a considerably high reliability.

Independent Variables

The adoption of APCDs is used as a variable of primary interest to capture the impact of the policy adoption. To make a comparison between those who have adopted and those who have not adopted as starkly as possible, only the states that formally mandated the APCDs are considered as adopted and coded as “adoption=1,” while voluntary efforts or strong interests are treated as “adoption=0.”

Data availability is another variable of interest that captures the degree of a transparent environment. The Hospital Compare dataset, governed by the CMS, have been broadly obtained in collaboration with hospitals, employers, accrediting organizations, and so on. However, some organizations fail to report their performance. The availability of quality information was calculated by the number of hospitals that reported their quality metrics divided by the total the number of hospitals in a county. Data availability was computed separately depending on each dependent variable. For clinical quality, the data availability of HAIs and 30-day readmission rates are used, whereas for perceived quality, the data availability represents the degree of accessible data on patient satisfaction.

The Herfindahl-Hirschman Index (HHI) estimates the degree of market competition. The HHI is one of the most commonly used competition indexes by health care researchers. The HHIs are derived as the sum of the squared market shares of hospitals in the market based on hospital referral regions (HRRs). The market shares are calculated based on the number of beds in a hospital. Thus, a lower value of HHI indicates that hospitals are exposed to more competitiveness.

To make sure that the publicly reported data are accurate and fairly reflect hospitals’ performance, possible factors that are likely to influence their performance outcomes are controlled. The control variables included in the study include case mix index (CMI), urbanity, ownership, teaching affiliation, Medicare and Medicaid discharge rates (%), nonprofit market share, uninsured rate, and lobbying efforts. More detailed explanations of the variables are presented in Table 2.

Table 2 Factor Loadings for Variables

Data

This research analyzed the US data from non-profit and for-profit general acute care hospitals. The three major data sources include: the annual hospital survey conducted by the American Hospital Association’s (AHA), Hospital Compare published by the CMS, and the APCDs’ websites (www.apcdcouncil.org). Other supplementary data include: Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey for patient satisfaction, the Health Resources & Services Administration (HRSA) for the percentage of uninsured population, and the Center for Responsive Politics for measuring lobbying efforts.

It should be noted that the periods of data collection differ among quality measures. HAI and satisfaction-related data are collected and published annually, while the annually released readmission data are based on hospitals’ three-year performance. Specifically, this study incorporated the HAI and patient satisfaction data, collected in 2017 and released in 2018. The readmission data used in the model were collected from July 2014 to June 2017, and disclosed in 2019. All the independent and control variables were lagged by one year. As a result, the base year for independent and control variables in the HAI and patient satisfaction models was 2016, and 2014 for the readmission model. The total number of samples are 3221 for HAIs, 2529 for readmissions, and 3183 for satisfaction.

Methods

After the factor analyses for constructing the dependent variables, the three quality models were compared by running ordinary least squares multiple regressions using STATA 14.1. There was no sign of multicollinearity, but heteroskedasticity was detected by the Breusch-Pagan/Cook-Weisberg test. The existence of heteroskedasticity was remedied by using robust standard errors. The robustness test of the models was performed through sensitivity checks to make sure that the research results were credible, and the overall results remained largely the same and none of the combinations led to any conflicting conclusions from the original models.

Results

Descriptive Analysis

The current research examined the quality of care of general acute care hospitals across all the US states in 2017. Table 3 shows the descriptive results of the quality models. There are two different data years, 2014 and 2016. The data years were separated because the data on readmission were measured during the three-year performance period (July 2014 – June 2017), while the two other measures were obtained annually (January – December 2017). The overlapping period was year 2017. Assuming that it may take some time for the impact of transparency to get into full swing, the authors decided to lag the independent and control variables for at least 1 year from the year when the quality measures were collected. The independent and control variables used for the readmission model represent year 2014, and those used for HAIs and satisfaction models represent year 2016.

Table 3 Descriptive Analysis of Quality Model

The descriptive analysis table indicates that there is no difference in the number of APCD adopted states. Data availability on readmission (74.3%) and satisfaction (76.7%) are higher than that of HAIs (42.7%). The mean of HHI is 0.026 for both years. Hospitals maintain an average CMI of 1.5 and the mean bed size is about 190. Approximately 14% of hospitals operate in the 100 largest US 100 cities. More than one quarter of them are affiliated with medical schools. Hospitals generally tend to admit more Medicare patients (50%) than Medicaid patients (18%). Over 80% of beds in the market belong to nonprofit hospitals on average. The uninsured rate dropped from 13.3% in 2014 to 9.86% in 2016. Lobbying dollars of approximately $160,000 were circulated on average for both years. The amount varies from zero to over one million dollars.

Regression Analysis

Table 4 presents the regression analysis of each model: HAIs, readmissions, and satisfaction. Significant differences between hospitals’ care quality depending on APCD adoption were found in both clinical and perceived quality, when HHI is controlled. It is reported that hospitals in the APCD-adopted states are significantly associated with higher HAIs, while leading to lower patient satisfaction. The results may indicate the possibility that states seeking improved quality of care were likely to adopt the state-wide transparency tool. Regarding the impact of the availability of patient satisfaction data, it signifies a positive association. This is consistent with the hypothesis that a transparent climate can contribute to customer satisfaction. The publicly available hospital-specific satisfaction data may have encouraged hospitals to identify new targets for quality improvement.²⁹

Table 4 Multiple Regression Analysis of Quality Model

The relationships between data availability and HAIs as well as readmissions were statistically significant and positive, which raise similar concerns about the unintended consequences of public reporting in the previous literature. A possible reason why hospitals under more transparent environments are associated with lower clinical quality may stem from risk aversion problems invited by these transparency efforts.^7,9 An empirical study demonstrates that higher-risk patients are more likely to seek out high-performing hospitals when hospital-specific data are publicly available.⁹ In addition, hospitals may strategically avoid more difficult to treat or severely ill patients who would negatively affect their performance on quality at least in the short term, which eventually deteriorates social welfare.³² A recent survey conducted by the Commonwealth Fund and the Kaiser Family Foundation also illustrated that half of the primary care providers viewed that the dissemination of quality measures had lowered the quality of care.³⁰ Another plausible reason for this result stems from the fact that quality improvement following the launch of public reporting was achieved mainly by high-scoring facilities, while their low-scoring counterparts showed no change or worsened their quality of care.³¹ As Werner (2009) found, the widened quality gap between high-performing and low-performing providers along with publicly reporting quality information may be attributable to the higher HAIs and readmission rates.

The findings on the relationships between HHI and quality measures demonstrated conflicting results. While observing HHI’s positive and significant relation to HAIs and readmission rates (β = 1.647, p < 0.01 and β = 2.134, p < 0.01 respectively), another significant and positive relationship between market concentration and patient satisfaction was found (β = 3.544, p < 0.01). Given that higher HHIs represent more market concentration and less competitiveness, hospitals in a more concentrated (ie, less competitive) health care market were likely to result in higher rates of HAIs and readmissions, supporting the traditional theory of competitive markets.^19,21 In contrast, the results demonstrate that a higher degree of market concentration tends to increase customer satisfaction. The interwoven relationship between APCD and HHI also showed that higher market concentration is related to higher patient satisfaction in the APCD-adopted states.

In the evaluation of the role of competition toward mediating the role of transparency and quality outcomes, it is found that market competition plays a significant role in deciding the impact of transparency on clinical quality outcomes.²² Based on the statistically significant and negative association among clinical quality metrics, data availability, and HHI, it is interpreted that hospitals located in a less competitive market (a higher HHI) presented the possibility that increased data availability can contribute to improving clinical qualities, represented by the reduction of HAIs and readmissions. Meanwhile, hospitals’ perceived outcome showed a disparate association with transparency and HHI. The relationships among perceived quality, data availability, and HHI implied that data availability was negatively linked to patient satisfaction when it comes to hospitals operating in a less competitive market. Hospitals’ actual performance may have marginally changed or failed to satisfy better-informed patients’ expectations for the medical services.^32,33 This is in contrast to the associations between market competition and two clinical quality outcomes under the rule of transparency. It is assumed that the contrasting responses have occurred due to the two disparate types of quality measures.³⁴ For example, Propper et al argued that hospitals in competitive markets sacrifice intangible quality outcomes (eg, satisfaction) for easily observed hard numbers, such as HAI and readmission rates in this case.³⁵ Also, providers may not expect that efforts toward better customer satisfaction will be compensated by higher prices, increased volume, or decreased costs, while clinical performance will.³⁶ Regardless of the results, however, it is indisputable that market structures have a substantial influence on quality of care.^35,37–39

Discussion

Despite the theoretically acknowledged merits of transparency, the present study noted that its ability to enhance quality of care remains inconclusive, and the pursuit of transparency may even inadvertently harm quality of care.⁴⁰ The disclosure of hospital performance information was found to have the power to differentiate hospitals’ clinical and perceived quality, but it failed to reach the desired outcomes without market pressure. To specify, hospitals in the states that have deployed APCDs showed an increase in HAIs as well as a decline in patient satisfaction. Having APCDs or maintaining more available data did not automatically result in the intended policy impact.^41–43 The results may also indicate the possibility that states with the need for improved quality metrics tend to adopt the transparency tool.

It is noteworthy that the impact of transparency on quality of care differs in terms of two factors: the characteristics of transparency policy and the quality metric types. First, the two public reporting initiatives have different levels of public awareness that derive from the supremacy of ownership. Hospital Compare data, managed at the federal level, are more widely acknowledged by over 4000 Medicare-certified health care providers than by APCDs.¹⁷ Moreover, states’ efforts on health care transparency can be stricken by federal decision, as the legitimacy of state-mandated reporting of claim files to APCDs has been threatened by the Supreme Court. In Vermont, for example, the Court invalidated the state’s APCD statute that mandated reporting of health claims data from self-insured health plans because it imposed duties that were inconsistent with the central design of the Employee Retirement Income Security Act (ERISA). Therefore, it is plausible that hospital managers have strategically differentiated their organizational behavior in response to the two public reporting initiatives.

In addition, scholarship in management stressed the importance of incentive schemes because different incentive systems result in organizational behavior consequences.^44,45 The two transparency approaches differ especially in terms of extrinsic motivation. Public profiling governed by CMS is closely tied to financial incentives, whereas APCD is not. Medicare’s Hospital Compare data are the standardized measurements for the federal government uses to reimburse the health care service costs of Medicare participating hospitals. Thus, hospitals are highly likely to meet the criteria in the performance measurement metrics, such as quality, safety, efficiency, or patient satisfaction, while APCDs are deployed and implemented at the discretion of a state, and not necessarily linked to a state’s payment system. One suggestion for improving the efficacy of APCD is to tie the state-mandated reporting to the state Medicaid payment system.

This research reported that the relationship between transparency and market competition for clinical quality outcomes consistently diverged from that of perceived quality. The inconsistency and complexity between clinical and perceived quality measures have created challenges for healthcare providers.⁴⁶ While measures of patient experience capture certain quality metrics, some situations where high-value care is at odds with patient satisfaction may exist.³⁴ For instance, high patient satisfaction was associated with inpatient utilization, higher healthcare spending, and higher mortality.⁴⁷ Seemingly unintuitive, it is addressed that “sicker patients may be both more satisfied and more likely to die” because greater consumption of resources tends to result in a better healthcare experience (p. 1113).⁴⁸ As a result, patient perceptions are often not quite aligned with clinical quality.⁴⁹

None of the quality measures can be simply ignored because underperformance on any of these measures would bring a considerable financial risk to hospitals as they are closely tied to government reimbursement, under the name of the value-based purchasing (VBP) program by CMS.⁵⁰ While hospitals may need to finetune their strategies for each quality measurement in order to cope with the new environmental pressure, further reexamination is imperative in order to discover whether those performance measurements are working in congruence with each other. It is the role of health policymakers to coordinate these quality metrics and improve the validity of patient experience measures and surveys.

This study is not free from limitations. The current research did not include sufficient focus on consumers’ selection pathway, which encourages consumers to select health care providers that pay careful attention to providing high quality of care.^3,18,27,51 As shown in the Bristol Heart Scandal, the revelation of data can lead to enormous changes through public and patient engagement. It contributed to an improvement in the mortality rate for children’s heart surgery through better communication and honesty in the United Kingdom.⁵² It is also known that even extreme honesty about adverse events, unavoidable in medical management, is beneficial to the healthcare providers in a broad sense. Not only do patients want open communication with physicians even about small errors but also this openness contributes to reducing the occurrence rate of substandard care or malpractice loss.^53,54 Nonetheless, there is no solid evidence that consumers use the information given to make informed health choices.⁵⁵ More research is needed to investigate whether consumers actually use the comparative data, what hinders consumers from using the data, what information they find useful to make wiser choices, how to provide correct information, and what ways inequality in access to healthcare data can be eliminated.