Unveiling the Negative Customer Experience in Diagnostic Centers: A Data Mining Approach

Introduction

The healthcare sector, one of the fastest-growing industries worldwide, faces challenges due to globalization, including addressing customer experience.^1,2 Customer experience is shaped by prior knowledge and interactions.^3,4 While technological advancements have generated vast amounts of data to explain customer experiences, organizations struggle to translate this data into actionable models and comprehend customer behavior.⁵ Diagnostic centers, a key component of the healthcare system, produce large volumes of data, necessitating the storage and analysis of customer experience information by healthcare providers.^6,7 Technology can enhance healthcare management by streamlining processes and improving efficiency in data analysis related to customer experience.^8,9 Electronic Health Records (EHRs) facilitate documentation and coordination.¹⁰ Data mining techniques provide insights into valuable customers and their experiences.¹¹ Additionally, data mining can help identify patterns in complaints and inform solution development. However, managing the sheer volume of data presents a significant challenge.

A prominent clinic, one of India’s largest integrated healthcare systems in the private sector, serves as the foundation for this research. It is named here as Beta clinic. The use of case studies to build theory and their advantages have been extensively discussed.^12,13 Each case functions as an independent experiment, establishing its analytical value. As a result, the case study approach is an inductive methodology that can be generalized. A thorough examination of the complaint volume reveals an unabated rise, necessitating the establishment of a comprehensive and robust complaint system.¹⁴ The increased usage of digital platforms and social media has fostered greater connectivity between customers and companies. Websites and consumer forums have empowered customers to register complaints against any company.¹⁵ Consequently, thousands of complaints are logged daily on consumer forums, where customers anticipate problem resolution. This paper proposes to identify the major areas in which complaints arise due to unfavorable customer experiences, utilizing a database of complaints containing complaint titles and complaint text. This research endeavor is a novel attempt to employ data mining techniques to decipher customers’ experiences in diagnostic centers based on their complaints. To effectively address the research objectives, the following key measurable questions are posed:

1. What are the primary categories of complaints that contribute to unfavorable customer experiences at the Beta clinic?
2. What are the critical factors that lead to unsatisfactory customer experiences?
3. Do customers from different locations exhibit similar experiences at the Beta clinic?

The remainder of this manuscript is structured as follows: Data and Methodology delves into the existing body of knowledge related to customer experience in healthcare settings; Results deals with data and methodology; Discussion presents the key findings derived from the data analysis; Conclusion engages in a thorough discussion of the results obtained, delving into the implications of these findings for both theoretical understanding and practical applications in enhancing customer experience within the healthcare sector, and also proposes directions for future research; the final section 6 summarizes the main contributions of this study, and proposes suggestions in the area of customer experience in healthcare settings.

Review of Literature and Research Gap

Recent research emphasizes the importance of patient-centered care and customer experience in the healthcare industry.^16–18 Positive customer experiences lead to long-term relationships and improved customer satisfaction.^19,20 However, many healthcare systems prioritize customer care over customer experience.²¹ Smart technologies can enhance customer experience²² and there is a growing focus on improving customer experience in developing countries.^23,24 Prior research suggests that organizations should utilize digitalization to gather and analyze customer feedback for co-creation.²⁵ Complaint data should be comprehensively analyzed to identify areas for improvement.^26,27 Data mining techniques can be employed to investigate the relationship between system deficiencies and customer complaints.^28–31 While data mining has been applied in hospitals for various purposes, there is a lack of studies specifically focused on diagnostic clinics, which generate a substantial amount of customer service and complaint data.^32–36 This research aims to address this gap by analyzing customer complaints from Beta Clinics, a diagnostic center chain.

Customer Experience

Patients and their accompanying family members or attendants constitute the customer base of diagnostic centers, distinguishing them from customers in other businesses. These individuals often encounter heightened anxiety, tension, stress, and negativity³⁷ and must feel satisfied with the quality of medical services provided within the available resources.³⁸ Given the connection between customer experience and cognitive and emotional functions,³⁹ handling such clientele demands special care and attention to ensure a favorable customer experience. This, in turn, encourages not only repeat visits from existing customers but also referrals to the diagnostic center.³⁷

Customer Complaints

Customer complaints serve as a crucial reflection of overall consumer experiences, particularly unfavorable ones.⁴⁰ By effectively utilizing customer feedback, organizations can enhance their overall performance and consequently improve customer experience.^41,42 Complaints can originate from both former and current customers, and they should be taken seriously and addressed promptly.⁴³ When analyzed appropriately, customer complaints can be viewed as an opportunity to retain existing customers or attract new ones.⁴⁴ Moreover, addressing customer concerns not only enhances customer satisfaction but also upholds ethical standards in the marketplace.⁴⁵

An extensive review of the literature reveals that customer experience, customer complaints, data mining techniques for analyzing customer complaints, and customer experience in the healthcare industry have been extensively studied as individual topics. However, a significant portion of the empirical research relevant to this topic has been conducted outside of India. While data mining has been employed in various sectors, its application in diagnostic centers remains limited. Similarly, research specifically focused on analyzing customer complaints to understand customer experience in diagnostic centers is scarce. Despite the efforts of researchers and corporate organizations in studying customer experience, there is still room for further academic investigation in this area.⁴⁶ This research aims to address these gaps by utilizing data mining techniques to uncover customer experiences in diagnostic centers. Therefore, to bridge the aforementioned research gaps, it is essential to examine customer complaint data through data mining techniques to gain insights into unfavorable customer experiences in diagnostic centers.

Data and Methodology

This research utilizes data from complaints filed against a prominent clinic on the consumer complaints website www.consumercomplaints.in. For anonymity, the clinic’s actual name is concealed, and a pseudonym “Beta” is employed. Analyzing complaints manually can be time-consuming and laborious. Data mining tools, on the other hand, can expedite knowledge-driven decision-making by automating and supervising data analysis.⁴⁷ The Apriori algorithm is an effective tool for mining frequent datasets and associated association rules.⁴⁸ Researchers have employed this algorithm for data mining to uncover hidden patterns and insights from massive databases.^49,50 Srikant and Agrawal⁵¹ developed the Apriori algorithm used to analyze complaint texts and titles separately to identify the primary complaint categories. The frequency of various terms used in complaints is also analyzed to create word clouds, visually representing the most prevalent terms. This paper’s primary objective is to employ data mining techniques to evaluate customer experiences at Beta Clinic. The customer experience database is derived from online complaints registered on the clinic’s portal. To analyze patterns in the data, this paper utilizes the Apriori algorithm proposed by Agrawal and Srikant.⁵²

Data Collection

Complaints were collected using web scraping with Selenium, BeautifulSoup, and Pandas in Python on an Ubuntu 20.04 machine. The dataset consists of 2096 complaints from various Beta Clinic centers across India. The complaints were collected and stored in spreadsheets having two columns namely “Title” and “Complaint Text”.

Preprocessing of Complaints

The complaint data undergoes several preprocessing steps to clean and prepare it for analysis:^53,54

1. Character Removal: Unwanted characters are removed.⁵⁵
2. Tokenization: Complaints are split into individual words or phrases.
3. Stop Word Removal: Common words with little meaning are removed.⁵⁴
4. Lemmatization: Words are reduced to their root form.^56,57

These steps ensure meaningful data for further analysis and modeling.

Data Analysis

The data analysis is explained in the following paragraphs:

The Apriori Algorithm

The Apriori algorithm, developed by Agarwal and Srikant,⁵² is a widely used method for frequent pattern mining, particularly in transaction databases.⁵⁵ Its key concepts and term definitions are as follows:

• Item: A unique object in the transactions database. In this context, each unique word in the list of complaints is considered an item.
• Transaction: A set of distinct items. In this context, the list of words from each complaint after preprocessing constitutes one transaction.
• Support: The total number of times an item appears in the transactions. It is often expressed as the ratio of the total number of occurrences of an item to the total number of transactions.
• Minimum Support: The minimum supports an item must have to be considered frequent. In this context, the minimum support of a word is the total number of complaints it appears in, divided by the total number of complaints.
• Candidate Item Sets: Item sets with a fixed number of items, often denoted by Ci, where “i” represents the number of items in the item set.
• Frequent Itemsets: Item sets with support greater than the minimum support, often denoted by Li, where “i” represents the number of items in the item set.
• Apriori Property: Any subset of a frequent itemset must also be a frequent itemset.

The Apriori algorithm employs a level-wise search to generate item sets. It starts with item sets containing one element and progressively generates item sets from lower levels to higher levels. The algorithm for generating C₁ is presented in Algorithm 1 (Appendix 1).

To generate candidates for the L_k item set, the L_k−1 item set is used. C_k, a superset of L_k, is generated by performing a set join operation on L_k−1 with itself. Algorithm 2 (Appendix 2) describes the procedure for generating C_k.

C_k is pruned using the Apriori property to eliminate candidate itemsets whose subsets are not frequent itemsets. Subsequently, the entire transaction data is scanned to identify item sets in C_k with a frequency exceeding the minimum support. Itemsets with support less than the minimum support are removed from C_k, resulting in the formation of L_k. Algorithm 3 (Appendix 3) details the process of scanning the data and generating L_k. This process iterates until no further item set with support greater than the minimum support can be generated.

In this study, the Apriori algorithm is utilized to identify frequently co-occurring words in the complaint data. These frequent itemsets are then analyzed for association rule mining to uncover closely related words in the complaints.

Association Rule Mining

Association rules capture the dependency of one data item on another within a dataset.⁵⁸ An association rule comprises an antecedent and a consequent. The antecedent represents a data item present in the data, while the consequent represents a data item that frequently co-occurs with the antecedent. The strength of this association is measured by confidence, which is the ratio of occurrences of the antecedent and consequent together to the occurrences of the antecedent alone.⁵⁹ Confidence values range from 0 to 1, with 1 indicating a strong association and 0 indicating no association. For this analysis, only rules with confidence greater than or equal to 0.7 were considered significant.⁵⁹ Algorithm 4 (Appendix 4) describes the procedure for mining association rules from frequent itemsets.

Word Cloud Generation

Frequent words were visualized using word clouds to represent their frequency in the data. Word sizes correspond to term frequencies, with larger words indicating higher frequencies.⁶⁰ In this study, word clouds were generated from complaint data using the “wordcloud” and “matplotlib” modules in Python.

Results

Both the complaint title and the complaint text were analyzed separately to identify frequent words and association rules. This approach was taken because the title often encapsulates the essence of the complaint text. The findings from both the title and complaint text analysis are discussed in the following subsections.

Analysis of Title of the Complaints

The complaint titles were analyzed to identify frequent terms and association rules. Table 1 presents a list of single terms with their corresponding support, calculated using a minimum support threshold of 0.02.⁶¹ This means that terms appearing in more than 2% of the complaints were selected for analysis. The frequency of words in Table 1 indicates that approximately 15.8% of complaints were related to reports, highlighting a potential area for improvement in customer experience. Notably, 14.7% of complaints were marked as resolved, suggesting a positive aspect of customer experience. Additionally, 15% of complaints concerned poor service quality, while 7.3% addressed staff behavior. Interestingly, 3.2% of complaints were related to COVID vaccination, and the same percentage of complaints used the terms “misleading” and “negligence” to describe the experience.

Table 1 Words with a Maximum Frequency in the Title of Complaints

After analyzing the most frequent words, a group of two words with maximum frequency were calculated and the result was tabulated in Table 2.

Table 2 Group of Two Words with a Maximum Frequency in the Title of Complaints

The data from Table 2 indicates a significant portion of customers (6.3%) have experienced difficulties obtaining test reports. Additionally, 4.2% of complaints specifically mentioned poor service as the source of their dissatisfaction. Furthermore, 2.1% of complaints cited specific issues such as rude behavior, medical negligence, misleading staff, lengthy wait times, problems with vaccine certificates, and incorrect test reports as contributing factors to their negative experiences. The data further reveals that the JP Nagar clinic accounts for the majority of complaints (3.2%), followed by the Salt-Lake clinic (2.1%).

A group of three words that occurred together in a complaint has been tabulated in Table 3. The minimum support used to calculate the frequency of the terms was 0.02.⁶¹

Table 3 Group of Three Words with a Maximum Frequency in the Complaints

Table 3 shows that 3.1% of complaints were related to JP Nagar clinic while 2.1% of the complaints were related to Salt Lake clinic and included the term resolved. 2.1% of total complaints indicated problems in receiving test reports.

Table 4 contains groups of four words that occur in a single complaint. The minimum support used to calculate the frequency of the terms was 0.02.⁶¹

Table 4 Group of Four Words with a Maximum Frequency in the Complaints

Table 4 reflects that about 2.1% of the complaints termed their experience as the worst and these complaints were related to JP Nagar. The same number of complaints were also marked as resolved.

In Figure 1, a word cloud has been generated to visually represent the frequency of the different terms in the title of the complaints. In Figure 1, the size of the words signifies the frequency of the terms. The more the frequency of the term, the larger the size.

Figure 1 Word cloud generated from complaint title.

After analyzing the frequent terms, association rules were generated using association mining. The most significant rules are tabulated in Table 5. The minimum confidence level was specified at 0.7.

Table 5 Rules Extracted from the Title of the Complaints

The Table 5 shows that whenever a delay is mentioned in a complaint, a report is also mentioned. Additionally, all complaints registered for the Salt Lake clinic and JP Nagar clinic have been resolved by the management. This suggests that there may be a link between delays and reports, and that the management is taking steps to address complaints.

Specifically, the table shows the following:

• “delay” and “report” have a confidence of 1.0, which means that whenever “delay” is mentioned, “report” is also mentioned.
• “Salt Lake clinic” and “resolved” have a confidence of 1.0, which means that all complaints registered for the Salt Lake clinic have been resolved.
• “JP Nagar clinic” and “resolved” have a confidence of 1.0, which means that all complaints registered for the JP Nagar clinic have been resolved.

An analysis of complaint texts revealed the most common terms and association rules. Table 6 lists the single terms and their corresponding support values, with a minimum support of 0.02. Sorting the terms by support highlights the most frequently used terms. The data indicates that “test” (27.4%) and “report” (15.8%) were the most common terms, suggesting that 15.8% of complaints concerned test reports. “Service” (12.6%) was also prevalent, while “behavior” (3.2%), “unprofessional” (3.2%), and “pathetic” (3.2%) were used less frequently. This implies that at least 12.6% of complaints were related to poor service.

Table 6 Words with a Maximum Frequency in the Complaint Texts

The most frequent two-word terms in complaint texts were identified and tabulated in Table 7. The minimum support used to calculate term frequency was 0.02. Terms were sorted based on support for analysis. Table 7 shows that “test report” (12.6%) and “health checkup” (4.2%) were the most frequent two-word terms, indicating issues with these services. Additionally, “experience bad” (4.2%) and “appointment refused” (2.1%) were prevalent, suggesting problems with customer experience and appointment scheduling.

Table 7 Group of Two Words with a Maximum Frequency in the Complaint Texts

The most frequent three-word terms in complaint texts were identified and tabulated in Table 8. The combination of “report”, “blood”, and “test” appeared in 4.2% of complaints, with 1% specifically mentioning “wrong”, highlighting challenges in obtaining blood test reports.

Table 8 Group of Three Words with a Maximum Frequency in the Complaint Texts

A word cloud in Figure 2 visualizes the frequency of terms in complaint texts. Larger words indicate higher frequency. Association rules generated from complaint texts provided no significant insights.

Figure 2 Word cloud generated from complaint text.

Discussion

An analysis of complaints reveals that 12.6% of unfavorable experiences concern delays and errors in test reports, while 4.2% and 2.1% pertain to health checkups and COVID vaccinations, respectively. Delays in receiving test reports warrant serious attention. Such delays can cause patients undue suffering and exacerbate their ailments.⁶² It is crucial to recognize that delays or errors in test reports can lead to misdiagnosis or inaccurate assessments during patient follow-ups. Consequently, procedures that combine error detection with a search for potential root causes are necessary to implement preventive and corrective measures. Customers also reported unfavorable experiences related to staff behavior, including “rude behavior” and “poor service” (4.2% each) and “medical negligence” (2.1%). Rudeness significantly impacts healthcare service delivery.⁶³ Rude language and unpleasant behavior among diagnostic service providers jeopardize patient safety and the quality of care they receive.⁶⁴ Frow & Payne⁶⁵ found that customer satisfaction declines when services fall below expectations. Poor service leads to dissatisfaction, and unsatisfied customers are more likely to engage in negative word-of-mouth, switching providers, and complaining.⁶⁶ Poor provider attitudes can deter service utilization and foster low expectations and discriminatory behavior among healthcare providers.⁶⁷ This is going to affect the mindset of the consumers towards the particular diagnostic centre negatively. Geographically, customers from JP Nagar and Salt Lake reported the highest frequency of unfavorable experiences. Healthcare marketing policymakers should prioritize recruiting frontline staff with strong interpersonal skills and expertise to create a memorable customer service experience.⁶⁸ This study contributes to the existing body of knowledge in customer experience by examining unfavorable customer experiences in diagnostic centers, as reflected in customer complaints. The findings have managerial, policy, and theoretical implications.

Theoretical Contributions

This study’s empirical findings contribute to the existing research by highlighting the strong correlation between customer complaints and customer experiences in diagnostic centers, adding to its novelty. Additionally, it demonstrates the impact of data mining technologies on traditional customer relationship management processes in healthcare settings, enabling evidence-based assessment and effective outcomes. Comprehensive customer experience data will foster research-driven support for the growing field of customer experience in developing nations.⁶⁹ Overall, the study’s findings validate the customer experience scale in the context of diagnostic centers, advance theoretical understanding of the experience concept in healthcare, and provide valuable insights for diagnostic center marketers. The proposed framework in Figure 3 arises from the research conducted. Customers visiting diagnostic centers may have positive or negative experiences. Negative experiences can lead to three possibilities: repeat complaints, legal action, or switching to alternative providers. Customer complaints should be analyzed for categorization and subsequent investigation. This effort can help reduce customer attrition to alternative diagnostic centers. Data mining techniques employed in the study identified crucial factors such as rude behavior, negligence in report delivery, delayed reports, checkup issues, and poor service. Some factors stem from employee behavior, which can be improved through proper recruitment, training, etc., while others point to infrastructure deficiencies. The goal should be to analyze and transform these complaints into positive customer experiences. These findings can assist managers in enhancing customer experience management and delivering superior, memorable experiences.

Figure 3 Proposed model of Customer Experience in diagnostic centers.

Managerial Implications

Enhancing customer experience in diagnostic centers requires a multifaceted approach that encompasses complaint management, data-driven decision-making, and customer-centricity. Howarth et al¹⁴ advocate for viewing complaints as opportunities for improvement, emphasizing their role in customer retention and reputation management.⁷⁰ Data-driven decision-making, as highlighted by Satish & Yusof,⁴² necessitates employee training to effectively utilize customer data.⁷¹ Framing patient experiences as customer service issues encourages staff to prioritize interpersonal skills and approach patient concerns as communication challenges.⁷² The areas of improvement are quite evident from the online feedback⁷³ and ratings, so corrective measures can be taken accordingly. Holmlund et al⁷⁴ emphasize the importance of quantifying the effectiveness of customer experience enhancement efforts, advocating for the use of key metrics to track progress and inform future decisions.

Adopting a customer-centric approach, prioritizing customer needs and perspectives, can significantly enhance patient satisfaction. Schiavone et al,⁷⁵ Ponsignon et al,⁷⁶ and Lee⁷⁷ highlight the benefits of this approach, suggesting that healthcare organizations can create more personalized, responsive, and effective service experiences. Managers should replace the restrictive legal paradigm for handling complaints with a management approach that views complaint resolution as a strategy for customer retention and organizational learning. Griffey & Bohan⁷⁸ and Nikitha et al⁷⁹ advocate for a proactive approach to complaint management, encouraging diagnostic center managers to embrace new data analytics technologies to investigate unfavorable customer experiences and identify root causes.

The ever-expanding range of technologies necessitates adaptability and a willingness to explore new approaches to service delivery.⁸⁰ Managers should utilize natural language processing and sentiment analysis techniques to transform unstructured patient experience reports into actionable metrics of healthcare performance, particularly in diagnostic services. Greaves et al²⁰ propose a method for extracting valuable insights from patient feedback. Adopting the roster method, as suggested by Singh et al,⁸¹ can effectively manage COVID vaccination-related issues, streamlining the vaccination process, reducing wait times, and minimizing patient inconvenience. Diagnostic center managers should implement a reliable and 24/7 customer experience solution to provide decision-makers with continuous access to actionable insights. Holmlund et al⁷⁴ emphasize the importance of real-time data availability for informed decision-making.

Policy Implications

To bridge the practitioner-academic gap in managing customer complaints in healthcare, this study proposes a multi-pronged approach. Customer complaints should be viewed as service failings rather than legal issues, with customer relationship managers employing interpersonal tactics to address concerns without assigning blame.^72,82 A centralized regulatory framework should supplement the management model to manage performance, review complaints, and respond to complainants.^38,83 This approach could enhance diagnostic service quality and empower patients to file lawsuits without waiting for an internal grievance process, expanding legal remedies for medical negligence. Governments should monitor the condition of hospital services and the overall health sector to ensure that citizens are treated as citizens first and customers later.⁸⁴

Directions for Future Research

This study proposes future research directions and provides examples of fruitful ways to explore the connection between customer complaints and customer experience. The Apriori algorithm, while an established tool, has limitations compared to newer data mining techniques. The study was conducted during the COVID-19 pandemic, and its findings may not reflect normal conditions. Additionally, the findings lack empirical validation through direct customer feedback. Future research should consider collecting data from consumers under normal conditions to validate these findings. Only 30% of businesses effectively utilize customer experience data to identify flaws and improve their market position.⁸⁵ Dissatisfied customers often switch providers and share negative experiences rather than communicating directly with the company.⁸⁶ Bolton et al⁸⁷ highlight the challenges in integrating digital, physical, and social realms to create superior customer experiences. Descriptive research is needed to better understand the customer experience that links offline customer complaints and experiences with service delivery. Further cross-sectional and longitudinal studies are required to generalize the findings.⁸⁸ Different variables, with different levels of importance, influence the customer experiences. Researchers can use the relationship between the parameters and social network analysis (SNA) to determine this association.^89,90

Conclusion

Errors are inevitable in healthcare, and healthcare professionals should accept this reality. Acknowledging that mishaps may occur and patients may complain should prompt practitioners to develop a complaint management strategy.¹⁴ Proactive mitigation is preferable to reactive complaint handling. Patient complaints should be viewed not just as post-consumption feedback but also as an opportunity to improve service delivery.⁸² A robust complaint handling system built on understanding, empathy, action, and honesty can benefit all parties involved.¹⁴ By collecting and aggregating patient descriptions of negative experiences online, patterns of poor clinical practice may be identified. Over time, this approach could help identify areas for improvement.²⁰ However, the internet is susceptible to false information, so separating reliable data from unreliable is crucial.⁹¹ Methods and analytics designed for managing and mining unreliable data must be employed to cope with erroneous and misleading information. Diagnostic centers should prioritize designing and implementing an effective complaint management system and ensuring their customers have a positive experience.