Selecting best-suited “patient-related outcomes” in older people admitted to an acute geriatric or emergency frailty unit and applying quality improvement research to improve patient care

Introduction

Our population is aging worldwide, and the proportion of oldest old is rising rapidly. The number of people aged 80 and over in the UK is expected to more than double to six million by 2037.¹ Older people are living longer, either leading to successful healthy aging or living with multimorbidity and/or frailty. More than 80% of older people above 85 years of age have multimorbidity and the mean number of morbidities is 3.62% ± 2.30.² 30% of older people >85 years could have physical–mental health comorbidity.²

Even though older people (aged 65 and above) comprise a relatively small proportion of all patients attending emergency departments, they constitute a much higher proportion of overall hospital inpatients.^3,4 An increasing proportion of acutely unwell older people with atypical presentation and complex needs, compounded by lack of expertise, results in higher conversion rates from the hospital front door to the wards. This, in turn, has resulted in a longer length of stay (LoS), a risk of deconditioning, and an impact of hospitalization.⁵

Geriatricians and general physicians have a key role in the provision of care for very old and frail patients with predominantly physical and mental health disorders.³ Current policies and guidelines are encouraging health care professionals to help people to age well by providing integrated services.³ However, clinical outcomes are largely published for individual diseases, and most randomized trials exclude older frail patients with multimorbidity.⁶ The units providing acute care to frail older people like acute care for elders,⁷ an emergency frailty unit (EFU),^3,8,9 or an acute geriatric unit (AGU)⁴ have been shown to provide cost-effective high-quality service. In this review, multimorbidity, frailty, the service model, and a wide range of “direct” and “indirect” patient-related outcomes of “at risk” older people admitted to an acute geriatric care unit are explored. In addition to data measurements, a quality improvement (QI) methodology to improve patient care based on the patient-related outcome data is discussed to ensure the effectiveness of our current services and enhance academic research.

Multimorbidity and frailty

Both multimorbidity and frailty are associated with aging and adverse outcomes. They impact not only on individual quality of life (QoL) but also on the caregiver. Any acute illness or trigger, like a fall, could lead to a hospital admission and rapid decline in function, and therefore, requires early identification and immediate attention. AGU care during the acute phase of older people’s illness or injury has proven to improve patient and system-level outcomes.⁴ The critical interventions to achieve optimal health outcomes for older people admitted to acute care include the following: 1) a team approach to deliver care either directly in a designated unit for older patients or indirectly using gerontological expertise in a consultancy model; 2) targeted assessment techniques to prevent complications; 3) an increased emphasis on discharge planning; and 4) enhanced communication among care providers across the care continuum.¹⁰ The components tested were based on best practice including patient-centered care, a prepared environment, frequent medical review, early rehabilitation, and early discharge planning. The Royal College of Physicians (UK) published Acute Care Tool Kit 3 in 2012 recommending the role of the right person for the right patient in the right place.¹¹ The hypothesis was that performing the assessment of older people early on in acute medical units (AMUs) has the potential to improve outcome, reduce inappropriate hospitalization, and reduce the need for long-term care.

Models of AGUs/services

There are several models of acute geriatric care, which could be based on age, need, or frailty.^12–14 Irrespective of the service model, the key principles of acute geriatric care must be aimed at: rooting out age discrimination, provision for Comprehensive Geriatric Assessment (CGA), and streamlining care by breaking down the barrier between medical and social care. There is very good evidence that CGA multidimensional assessment with multiagency assessment and management in older people leads to better patient outcomes.¹⁵ CGA has evidence for reduced readmissions, reduced long-term care, and lower costs.¹⁶ CGA emphasizes integrating QoL, functional status, prognosis, and outcomes with standard medical diagnostic evaluation.¹⁷ Therefore, a typical AGU/EFU team must comprise a geriatrician, a nurse specialist, an occupational specialist, a physiotherapist, a pharmacist, and others as needed. Various existing service models to meet the needs of acute frail older adults have their benefits and disadvantages.

Needs-related models

These models operate by identifying a focused population who may benefit from acute geriatric care based on set criteria such as nonspecific symptoms, cognitive and/or functional decline, multiple complex pathologies, and social care needs. This has the disadvantage of a need for expertise to identify the right patients for the right service.¹³

Age-defined models

Age-defined models seem to be justified in providing acute geriatric care on biological, screening, and pragmatic grounds. However, as age has not been shown as a valid screening tool to identify those who benefit from geriatric care, this model fails on medical grounds. In the UK, the Royal College of Physicians does not support age-discriminated services. On the other hand, units which avoid systematic ageism can create challenges in delivering CGA as the evidence base is for dedicated teams delivering CGA for older people.^13,14

Integrated models

These models involve geriatrician-led teams and have the advantage of access to a multidisciplinary team (MDT) including therapists. It is thought to be a rational response to the biological nature of aging and the pattern of health and social needs of an aging population. General Internal Medicine training in the UK has incorporated geriatric medicine as part of the curriculum, to improve the competence of generalists for managing older people in the acute hospital setting.¹³

Geriatric liaison services in AMUs

Integrated services with prompt geriatrician input in AMUs have shown improved patient outcomes.¹⁸ Despite the risk of duplication, coproduction between the AMU and the geriatric liaison teams has resulted in high-quality acute care with safe and timely discharges into the community.¹³

Patient-related outcomes

Outcomes are the differences or changes made through a service or an enhanced activity. Patient-related outcomes should determine wider impact and explore an issue from the service user and stakeholder perspectives but keep the focus on research and QI. Therefore, outcomes should be measured at different levels: patient (direct clinical outcomes); staff; family and carers; and community and organization (indirect clinical outcomes).

Clinical outcomes

Routine patient-related clinical outcomes including LoS, 30-day emergency readmission rate (ERR), number of days spent at usual place of residence in the 90 days after discharge, inpatient mortality, 30-day mortality, 90-day mortality, or 1-year mortality, discharge to own home, usual place of residence, or new care home, or institutionalization can be used to benchmark any acute geriatric services and can be compared against the national standard outcomes. Other clinical outcome data at the time of discharge include the need for a type of care package, increased dependency, or mobility at the time of discharge using a validated scale.

However, these clinical outcomes cannot be interpreted without appropriate patient characteristics and descriptive markers. Therefore, it is recommended that when planning clinical outcomes, efforts should be made to define the study population. Routine demographic markers including age, gender, companionship or marital status, residential status like living in flat, house, or a care home, and geographical location are routinely available.

The comorbidity burden can be defined as the number of total comorbidities or can be measured using Charlson’s Comorbidity Index.¹⁹ Patients can be described on the basis of their history of chronic cognitive impairment, or dementia, prior to acute admission. It is helpful to describe patients based on a number of drugs or polypharmacy. Similarly, frailty can be measured using Fried’s phenotype²⁰ and the Rockwood deficit model.^21,22 Frailty can be defined phenotypically based on the presence of three or more frailty indicators: slow walking speed, low grip strength, unintentional weight loss, subjective exhaustion, and low levels of physical activity. Frailty, based on these criteria, was predictive of poor outcome including institutionalization and death. Alternatively, a frailty index (FI) based on the accumulation of deficits with age, including medical, physical, functional, cognitive, and nutritional problems, can be used to define a study population. The FI expresses the proportion of the number of deficits from the total deficits considered in the study population and has been used in acute unwell older patients in an acute geriatric rehabilitation ward.²³ However, pragmatically, patients can also be defined based on the clinical frailty scale (CFS) level prior to admission, as measured by the 9-point CFS which is highly correlated (r=0.80) with the FI.²⁴

Patients can be described based on their functional performances using basic activities of daily living (ADLs) before admission to the hospital using validated scales, for example the Barthel Activities of Daily Living Index Scale^25,26 or the instrumental activities of daily living scale.^26,27 Other parameters to describe patients include the number of hospital admissions in the last 12 months; range of formal or informal carer support; and getting out of the house, for example, on less than or more than three occasions, with or without assistance.

In order to further generalize and compare outcome data from another AGU, older patients should also be described in depth at the time of admission. Various variables that can be measured on admission to an AGU include acute illness severity, as measured by the Modified Early Warning Score;²⁸ reason for admission; single-organ or multisystem failure; functional decline from baseline or 2 weeks prior to admission; nutritional status; skin breakdown; immobility or falls; and incontinence, presence of delirium, and dementia.

Staff and staffing-related outcomes

Considering current service pressures, medical complexity, and frailty, the nursing job is not only physically demanding but also emotionally challenging. Nurses may not be able to complete their work at their personal level of professional satisfaction. The skills and experience of a professional nurse are often poorly rewarded and extra hours spent may not be remunerated. This could lead to work-related stress, low morale, and unhappiness.

Less attention has been given to optimal professional practice, namely nurse education, continuity of care, and quality of the work environment.²⁹ Inadequate nurse staffing ratios of an AGU can have deleterious effects on patient outcomes. The quality of the nursing practice environment has been associated with nurse recruitment and retention, and nurses reported positive job experiences with a better care environment.^29,30

Health care needs of acutely unwell older people require an MDT approach in which team members should have an appropriate skill mix and knowledge about the aging process and associated complications.³¹ Regular nursing training is not only essential to provide enhanced care to acute older people but has also shown a significant reduction in stress from their routine workload in managing complex and frail older people.³² Nursing training and better-educated hospital nurse workforces are associated with lower 30-day patient mortality.^29,33

Therefore, it is essential that staff-related outcomes should be measured for comprehensive evaluation of the clinical outcomes of an AGU as there are well-documented associations between staffing and patient-related outcome. Various measurements in combination, including staffing ratios, nurse burnout levels, nurse job dissatisfaction or satisfaction level of the staff, work-related stress, needle-stick injuries, happiness level, nurses’ skill mix, sickness rate, recruitment and retention of staff, or gaps in the nursing rota, could be utilized for a comprehensive service evaluation.

Regular staff surveys about their work experience including having a yearly appraisal, satisfaction with the quality of work, discrimination, harassment, or abuse at work, the level of pay, flexible working, and equal opportunities at work can be used to compare various parameters over time.³⁴

However, for further research, a validated tool like the Practice Environment Scale of the Nursing Work Index (PES-NWI)³⁵ can be used to measure the nursing practice environment and its impact on the patient outcome.

Caregiver-related outcomes

Caregivers play an important role to support older people who have difficulties with their routine ADLs.³⁶ Carers could be formal and paid or informal and unpaid for this role. Carers’ perspectives on how well the staff were trained and informed are relevant to providing high-quality care, thus meeting the needs of an acutely unwell older person.

However, caregivers can have a physical, psychological, or emotional impact on their own health due to direct or indirect stress burden, a strain on life, or poor quality of their work–life balance.³⁷ Caregiving is an independent risk factor for higher mortality among caregivers as compared to noncaregiving controls.³⁸ In addition, caring for older people is not only challenging, but a heavy caregiver burden is also associated with high all-cause mortality and hospitalization for care recipients in community-dwelling older people. Therefore, reduction of the caregiver burden not only prevents deterioration of the caregiver’s own health but also avoids adverse clinical outcomes for older people receiving care.³⁹

The caregiver burden is often underrecognized by clinicians and hospital teams. It is recommended that physicians should also be responsible for recognizing and assessing the caregiver burden and interventions should be tailored to the individual circumstances and context to avoid or minimize the caregiver burden.⁴⁰ Caregiver outcomes are a type of patient-related outcome and should be included as part of an evaluation of an AGU. The caregiver burden can be measured using validated scales. The original 29-item self-reported Zarit Caregiver Burden Interview (ZBI) scale can be used to measure the subjective burden of an informal carer.⁴¹ The 22-item version of the ZBI is more commonly used in both routine clinical care and research.⁴² The 22-item ZBI, each question scored from 0 to 4, gives a score range from 0 to 88: 0–20=little or no burden; 21–40=mild to moderate burden; 41–60=moderate to severe burden; and 61–88=severe burden.⁴² 12-item and 4-item versions of the ZBI have also been validated for easier administration of the instrument.⁴³

Other scales that can be effective in evaluating the impact of the burden on many aspects of caregivers’ lives include the 24-item Caregiver Burden Inventory (CBI)⁴⁴ or a shorter 10-item Burden Scale for Family Caregivers.⁴⁵

Patient and community-related outcomes

Patients’ views are not optional but essential to achieve high-quality care.⁴⁶ High Quality Care for All,⁴⁶ Lord Darzi’s 2008 report, stated that the quality of patient care is defined in terms of three criteria: the effectiveness of the care, patient safety, and patient experience. Patient experience and care should be characterized by compassionate and dignified care, and their experience can only be improved by analyzing patient satisfaction with their own experiences.

Therefore, for an AGU to deliver ideal patient-centered care, it is essential to know the elements of compassionate care and their satisfaction. There is a wide variety of patient-related outcomes that can be used to develop an AGU on the basis of what patients care about, their well-being, and their QoL.

Patient-reported outcomes (PROs) are directly reported by the patients, based on their experience, without interpretation of the patient’s response by a clinician or anyone else. PROs define the patient’s QoL and the impact on their health and functional status following health care treatment.⁴⁷ PROs could play a key role to improve patient-centered care by measuring the impact of CGA, which is the cornerstone of an AGU.

In comparison, patient-reported outcome measures (PROMs) are the validated tools or scales to measure PROs and provide categorical (qualitative) or quantitative data. PROMs include a tool to define functional status or health-related QoL, symptoms, or disease-specific burden. PROMs assist in the evaluation of an existing service model or implementation of a new care provider.⁴⁸ Generic PROMs including EuroQol EQ-5D⁴⁹ and the 36-Item Short-Form Health Survey (SF-36)⁵⁰ are designed to be used with most patients but are less sensitive than disease-specific PROMs. Short Form 6-D (SF-6D)⁵¹ is an option.

Patient-reported experience measures (PREMs) are useful tools that are increasingly being used to obtain data on patients’ perceptions of their health and experiences while receiving care. PREMs allow management insight into patients’ expectations and could be very helpful in redesigning services with the aim of improving quality of care.⁵² PROMs are largely used by researchers but are not yet embedded in routine clinical practice to improve quality of care from the patient’s perspective.^52,53 Patients’ views should be incorporated into further service improvement and modernization, and current evidence supports that routine use of PROMs improves decision-making between doctors and patients and improves patient care.⁵⁴

Other scales that could be used to measure patient-related outcomes include the Hospital Anxiety and Depression Scale (HADS)⁵⁵ and the Geriatric Depression Scale (GDS).⁵⁶ The HADS is a validated scale that can be used as a screening tool to complete self-assessment of emotional distress including anxiety and depression. The score can range from 0 to 42, with higher scores indicating more distress which warrants further assessment. The score for each subscale (anxiety and depression) can range from 0 to 21 (normal 0–7, mild 8–10, moderate 11–14, and severe 15–21). Generalized anxiety disorder is common in older people and often underrecognized in the acute setting. It could be helpful to assess the psychological burden of the disease and associated fear or loss of confidence. A short-form GDS consisting of 15 questions can be used for depression, and a score over 5 should prompt further assessment.

Organizational-related outcomes

Patient-related outcomes have associations with the success of an organization and affect the organization’s performance, strategic model, and vision. Therefore, for a complete evaluation of an acute geriatric service model, their relationship with organizational-level outcomes should be measured.

Safe discharge planning and follow-up before discharge from an AGU with adequate involvement of the patient and the carer should be standardized and regularly monitored. The ERR and a reason for admission should be explored. Other outcomes that can be measured at the organizational level include complaints and litigation costs, hospital-acquired infections like Clostridium difficile, and pressure sores. Other adverse impacts of hospitalization include inpatient falls, as a higher falls rate has been reported from those with dementia.^57,58 The standardized incidence of the inpatient falls rate should be measured as the number of inpatient falls/1000 occupied bed days.⁵⁹ The impact of inpatient falls including inpatient hip fracture, discharge to a new care home, and mortality should be reported at an organizational level, ensuring standardized benchmark care of an AGU.

Often patients are deemed fit for hospital discharge by the MDT but the “medical fit” date could be earlier or later than actual discharge. Delayed discharges from an acute unit could increase the demand on acute geriatric services; therefore, exploring factors leading to a delayed transfer of care to the community should be regularly measured and compared within an organization. Screening of delirium, appropriate treatment, and communication of the diagnosis to community physicians should be consistently monitored at an organizational level. One-quarter of patients admitted to an AGU could have underlying dementia. Therefore, an appropriate dementia care plan and staff training could be recorded at the organizational level.⁶⁰

Patients admitted to an AGU undergo CGA which has proven improved clinical outcomes including discharge to their usual place of residence. A low rate of unplanned general practice (GP) consultations could be another organizational measure of excellence in service delivery. Access to social workers and intermediate care services for an AGU often helps early discharge from an acute unit and this should be monitored at an organizational level. A high carer burden is associated with adverse clinical outcomes; therefore, strategies and plans for carer assessment and engagement should also be analyzed at the organizational level.⁶¹

Data and measurements

This review introduces a brief guide to medical statistics. It provides an opportunity to understand, describe, summarize, and benchmark patient-related outcome data from an AGU in order to look for areas of improvement. Measurement plays a key role in all improvement efforts. Measurement is not just about creating charts, it is about understanding our services and systems. Measurement uses data and the purpose of such measures should be to understand processes to improve patient outcomes rather than just merely judging service outcomes. For detailed statistical analysis and tests, an appropriate statistical textbook or expert advice is recommended.

Describing and summarizing data

Any information, facts, or numbers that are collected to be examined, gaining information, and used to help decision-making is known as data. The aim should be to have benchmark baseline data and compare against those benchmark standards. Data should be presented in such a way that they do not lose their evidence and content. The best approach is to present data in terms of timeline series so that data do not mislead the user.^62–65

It is often impossible to describe the whole population, therefore a study sample for a service over a predefined time period should be used to describe population and benchmark service outcomes. Therefore, ensure that the study design includes a good subset of the population which is a close representation of the population sample to draw inferences, although it may not be most accurate to show all the characteristics of the population that a service covers. Statistics is the science of collecting, analyzing, interpreting, and presenting data through epidemiological studies and clinical or laboratory research trials. Statistics allow us to describe and summarize data by doing appropriate analysis and extrapolate from sample data to the whole population to draw valid and accurate inferences.^62–65

1. Types of data. Broadly, there are two types of data: categorical (qualitative) data and quantitative data. Categorical (qualitative) data include binary data (eg, yes or no, dead or alive); nominal (eg, home, care home); or ordinal (eg, mild, moderate, or severe frailty). Quantitative data include continuous data (eg, weight, body mass index, and FI) or discrete data (eg, number of inpatient falls, CFS). However, based on how data have been collected, it is sometimes possible for the same variable to present as categorical and quantitative data. Data can be cleaned to give meaningful results, for example pain could have been recorded as ordinal (mild, moderate, and severe) or discrete (eg, pain scale).^62–65
2. Distribution of data. The distribution of a variable or statistical data is a mean of describing the range of data, listing all the possible unique values of the data and the relative occurrence of each value that a variable represents in the dataset. Qualitative data are often represented using pie or bar charts. Continuous variables would be best grouped into bands and viewed in a histogram to show the relative frequency distribution, and the cumulative frequency will be 100% or 1. A histogram is also part of a normal distribution but can have several shapes: left or right-skewed distribution or implausible distribution. In comparison, categorical variables are best represented by a bar chart where data are shown as vertical blocks with the X axis showing a type of data and the Y axis representing a value for respective data. For example, place of residence before admission is best represented using a bar chart.^62–65
3. Spread of data. The spread of data, also known as a measure of dispersion, is used to describe the variability within a sample in conjunction with a central tendency such as the mean or median, to provide an overall description of the dataset. A normal distribution curve is a bell-shaped frequency distribution curve with a narrow variance where most data values are clustered around a central location (often mean) and is most representative of the sample. The mean is the average value in the dataset; the median is the value in the middle of the dataset in numerical ascending or descending order, whereas the mode is the most often occurring value in the dataset. An outlier is an observation which does not appear to belong to the data and could be incorrect data, but can be true but when cleaning data; be careful of outliers. For example, LoS is often represented as the mean but the median could be a better measure as it will not skew the data and outliers will be excluded naturally.^62–65

   Other measures of spread include the range (minimum to maximum values), quartiles (top 25%=Q1 and bottom 25%=Q3) and the interquartile range (middle 50%=Q3–Q1), variance (square of the variance of all the mean values), standard deviation (average deviance from the mean=square root of the variance), and standard error (a measure of the dispersion of the sample mean values around the population mean). A box and whisker graph is a simple way of representing a measure of dispersion on a plot in which a column is drawn to represent the interquartile range, and a vertical line inside the column represents the median value. Upper and lower quartiles are shown as horizontal lines on either side of the column.^62–65

4. Comparison of data. Probability is defined as the extent to which an event is likely to happen and is measured as a ratio of the possible number of times an event can occur to the whole number of all events. The degree of chance is between 0 (never occurs) and 1 (always occurs), and the sum of the probabilities of all possible events is 1. The best possible estimate is to insist on a high probability like 0.95, meaning 95% probability (clinical significance to give us the P-value). The confidence interval (CI) most commonly is 95% (point estimate between lower CI and upper CI). A good understanding of the right statistical test is needed when analyzing the data based on the type of data, paired and unpaired groups, and sample and a statistician’s opinion is recommended.^62–66

Presentation of data

Presentation of data refers to the organization of data into text, tables, or graphs so that true logical information and statistical conclusions can be easily understood and communicated efficiently from the collected measurements. A graph is a very effective visual tool as it displays data at a glance and facilitates comparison. For example, scatter graphs can be used to compare two sets of data where one set of data is put on the horizontal (X) axis and the other on the vertical (Y) axis.

Data can be presented as static data (snapshot at a particular time) or dynamic data (data over time). Static data can be difficult to interpret and could be judgmental whereas dynamic data tell the real-time story and effects of change can be easily interpreted. Therefore, for service improvement, graphs can also reveal trends and relationships within the data to demonstrate whether changes resulting in an improvement over time should be used.⁶⁷

Run charts and control charts⁶⁸ have been recommended for QI research. A run chart is a line graph where data are plotted over time against the median to understand the trends in the process and is an important tool for assessing the effectiveness of the change. Run charts are useful for tracking information and predicting trends or patterns. Run charts do not use upper and lower control limits. In comparison, a control chart, also known as a Shewhart control chart, is a statistical process control tool used to study how a process changes over time. Once sufficient data have been entered, a mean, an upper line for the upper control limit, and a lower line for the lower control limit are calculated. Control charts are used to analyze and understand process variables and monitor the effects of change on performance and outcomes. Control charts are more sensitive than run charts and predict process behavior more accurately by detecting common cause (stable process) and special cause (unstable process) variation.

Applying QI methodology to improve patient care

Although having benchmark standard outcome data is an achievement, generating better QI outcome data with time is the gold standard. Therefore, prepare baseline data, understand gaps in care, and think of change and which area needs immediate attention based on a QI collaborative methodology.

QI

Prudent health care principles set out by the Bevan Commission⁶⁹ suggest that “public and professionals are equal partners through co-production”, “care for those with the greatest health need first”, “do only what is needed”, and “reduce inappropriate variation”.

The Transtheoretical Model

The Transtheoretical Model^70,71 is widely used by professionals around the world for QI work. It is based on five stages of change to integrate the most powerful principles and processes of change.^70,71 This is often a key step to map team members and existing resources when planning to improve existing clinical outcomes and bring change.

1. The pre-contemplation stage is where team members are uninformed, faced unsuccessful attempts previously, do not intend to take action in the near future, are unmotivated, or are not ready to help.
2. Contemplation is the stage in which people intend to change in the near future. They are more aware of the benefits of change but not ready to act immediately although getting ready for a change.
3. Preparation is the stage in which people have participated in improvement work and intend to take action in the immediate future. These team members are ready, so should be actively invited for service improvement.
4. Action is the stage where team members have taken sufficient action in the past and adopted new practice but goals are not fully achieved, and with extra support the service could observe marked improvement.
5. Maintenance is the final stage where specific change has already happened and new healthier processes have been put into practice. Often, individuals recycle through the stages or regress to earlier stages from later ones. But stakeholder involvement, standardized policy, and wider awareness help to sustain the overall QI initiatives.

Process mapping

Process mapping is the central component of QI initiatives, and before implementing improvement work it is essential to understand various steps involved in any process completely and accurately.^72,73 Process mapping is used to capture all the steps and draw a sequential flow diagram involved in solving the problem or improving any particular adverse clinical outcome. While doing process mapping, get the team perspective, follow a patient or a situation, and look at and capture what really happens rather than what should happen. It is important to consider carefully the type of process mapping in the QI work, and it has been suggested to consider more than one map to ensure that different aspects of the process are captured.⁷⁴ The different process mapping measures include a “Pareto chart”, “fishbone mapping”, or the 5 Whys model.

For example, a Pareto chart is a type of bar chart in which all the factors that contribute to an overall effect are arranged in decreasing order according to the magnitude of their effect. The factor or factors with the highest impact warrant the most attention and help the team to concentrate and focus on a particular area.

Another way to complete process mapping is by using a fishbone diagram. This is also known as cause and effect analysis to identify causes of a problem, possible root cause analysis. For example, this could be used to identify a high inpatient falls rate in an AGU. Similarly, root cause analysis can also be done by repeatedly asking the question “Why” (often five times is considered a good rule of thumb), which can lead to the root cause of a problem.

Driver diagram

A driver diagram is often used to plan an improvement project. It is a diagrammatic illustration of a theory of change and outline strategies for various aspects of QI work in a systematic manner. In other words, a driver diagram is a visual strategy to tackle a complex problem. A driver diagram is just a representation of your QI strategies and may change with time.⁷⁵

A driver diagram has a measurable aim that needs to be achieved to improve an outcome measure, for example reducing the ERR for an AGU by 20%. The next part is primary drivers, which are the key areas that are needed in order to achieve an aim. So, capture key areas that will identify process measures and influence the aim. For example, two primary drivers to achieve aim could be a follow-up review within 4 weeks and 100% compliance with the discharge summary within 24 h of discharge from an AGU. Each primary driver could have a few secondary drivers which influence the primary driver.

Secondary drivers help to identify relevant change ideas, which can be tested in order to make a measurable change in the aim. For example, changes to be tested could be a new tick box on a discharge summary as a reminder to book a follow-up clinic in 4 weeks. The key part of a driver diagram is to identify which change would have maximum impact on the aim and then which would be easiest to introduce.⁷⁵

Models of improvement

There are various QI methodologies practiced widely in health care including Six Sigma,^76,77 Lean,⁷⁷ Health care Failure Modes and Effects Analysis (HFMEA),⁷⁸ and Plan–Do–Study–Act (PDSA) cycles.^79,80 PDSA is one of the commonly used models in health care to test an idea by testing a change on a very small scale or a specific population and measures its impact. It is very outcome focused and is based on the learning from previous cycles in a systematic way before implementing change on a wider scale.

PDSA methodology includes the following three key questions to answer before testing an improvement concept and a process for testing change ideas:^79.80

1. What are we trying to accomplish? (What is the aim?)
2. How will we know if the change is an improvement? (What measures of success will we use?)
3. What changes can we make that will result in improvement? (The change concepts to be tested.)

The PDSA cycle has the following four stages:

1. Plan – the change to be tested or implemented.
2. Do – carry out the test or change.
3. Study – based on the measurable outcomes agreed upon before starting out, collect data before and after the change and reflect on the impact of the change and what was learned.
4. Act – plan the next change cycle or full implementation.

It is always advisable to plan multiple cycles to test ideas and to test on a small scale. The idea should be implemented when most tests have been confidently completed and the best way of achieving the change has been agreed.

Measurements

Measurement for improvement provides feedback that changes are having an impact toward achieving the project aim (outcome), learns from PDSA cycles (process), and assesses whether the system as a whole is improving (balancing). Measurements also help us to understand a system, quality of care, reliability of processes, demand, and capacity. Measurement helps us to use data to complete service evaluation of existing and new services to focus on improvement and benchmark patient-related outcome. It is recommended not to use any improvement data to judge performances of other services. For example, comparing the inpatient falls rate of the unit against another unit and accountability for performance is not encouraged.

Applying QI to improve patient care

Quality initiatives based on the PDSA model of improvement have been employed and implemented in various services and have shown significant improvement in patient-related clinical outcomes.^81–85 A higher inpatient falls rate was observed in a hospital with 100% single rooms with en-suite facilities. A geriatrician-led, systematic nurse-training program on the understanding and correct use of an existing multifactorial falls risk assessment tool was developed using a PDSA methodology.⁸² The initial quality initiatives were aimed to raise awareness of nursing on routine ward rounds, daily board rounds, or nurse handovers.⁸² These initial quality initiatives did not show any improvement which led to the introduction of registrar-led nurse teaching which was later changed to systematic and structured consultant-led nurse training and was implemented in April 2013.⁸² These interventions achieved 58% reduction in inpatient falls in the pilot ward. Therefore, the quality initiatives were spread to the whole hospital which showed a significant reduction of inpatient falls by 34%.⁸² The subsequent evaluations of these quality initiatives have not only shown annual cost saving,⁸² reduction in the number of bed days,⁸² and reduction in hip fracture⁸³ but also a significant reduction in 6-month and 1-year mortality.⁸³

Multidisciplinary geriatric team members are managing very complex and frail patients with multiple conditions which bring new challenges. For example, in patients with Parkinsonism (PwP), osteoporosis is often underrecognized and undertreated. Considering the complex nature of Parkinsonism and human factors, quality initiatives were employed to measure the extent of undiagnosed and untreated osteoporosis. The study confirmed a high prevalence of fragility fractures of 22.6% in patients attending a Movement Disorder clinic.⁸⁴ Only 40% of PwP received evidence-based treatment for the underlying osteoporosis. QI work was undertaken by two consultant geriatricians, and case-based discussion (CBD) to assess falls and bone health for each patient with Parkinsonism with osteoporosis on a weekly basis was introduced.⁸⁵ Lessons were learnt and CBD was later developed into a monthly multidisciplinary meeting using PDSA methodology to include a physiotherapist and a Parkinson’s Disease Nurse Specialist. The results showed that following the introduction of quality initiatives, 91% of PwP were treated according to guidance: a 56% increase.⁸⁵ The improvement work was also spread to the community to ensure that all patients receive evidence-based treatment on an all-inclusive approach to reduce inappropriate variation and promoting partnership working with GP.⁸⁶ All patients who were deemed to be on inappropriate drugs, untreated, or at treatment failure were discussed in the consultant-led CBD with the general practitioner and practice staff virtually. The study concluded that one-third of patients with osteoporosis in a GP were not treated to guidance. As part of QI, evidence-based treatment was initiated, suggesting an overall improvement of 75%.⁸⁶

Discussion

Research is not just for researchers, every member of the team can do it. However, in clinical practice, most staff members appreciate research but it is very challenging for researchers to rely on clinical staff members to identify or recruit subjects.⁸⁷ Effective risk management is an essential part of most clinical consultations and benchmarking patient-related outcomes for an acute geriatric services highlights relative risks and benefits, thus improving the exchange of information between clinical teams and patients/carers.

Clinical research should be a part of routine clinical practice and clinical outcomes should be regularly monitored to ensure continuous service improvement by employing QI methodology. Clinical teams should be encouraged to ask good QI research questions and think critically about the answers. Teams would need to be relevant and fit for purpose clinical outcomes to measure improvement. The following three golden rules to measure clinical outcomes should be followed: ask or answer the right questions; focus on high-quality relevant data; and be systematic with communication.

QI teams can also test the alternatives through remodeling of care and service modernization. Rapid PDSA cycles facilitate new experience and learning, thus enabling testing a change and deeper understanding of the barriers for improvement. Small changes could lead to improved patient care and attract wider team and stakeholder engagement. Hence, evaluation of clinical outcome data demonstrates evidence of effective and efficient service changes rather than a mere opinion from service providers. There is a dearth of a wide range of patient-related clinical outcomes for AGUs, and most acute units have reported discharge to the usual place of residence or LoS which have shown similar results. Further research has been recommended to embed QI research in complex, multidisciplinary care for acute older patients.

Achieving change within our complex health care system is challenging and time-consuming. QI research is a team sport, and engagement of the whole team generates new ideas for change and allows teams to not only test new ways of working but also create ownership of the change. Engagement with a wider team and actively seeking out those who need to be involved at both an operational level and a strategic level is also needed to test new ideas to drive improvement. Every team member from the health care support worker, to senior nurses and consultant colleagues, through to managers, executives, improvement advisors, and even statisticians, and also the patients themselves should be valued as an individual and for their contribution to improving services. Therefore, key elements to lead QI research are a willingness to take an appropriate risk to seek new innovative ideas, working collaboratively toward a shared vision, and developing trusted relationships with high morale.

This review article has several strengths. We have explored a wide range of clinical outcomes from a patient, carer, stakeholder, and organizational level. We have discussed pragmatic ways to measure and improve these outcomes. Various service models and current evidence on CGA are not explored. We also acknowledge the lack of in-depth discussion on frailty and multimorbidities and ways to delay frailty as limitations of this review.

The key to the success of such a coproduction is to understand the purpose of the service and roles of the stakeholders, which is dependent on the needs of the establishment based on comprehensive patient-related outcomes. So, it is important to select and establish a suitable model of acute geriatric care agreed on local criteria and allocate resources to complete the outcomes and apply QI research methodology, as explored in this review.

Conclusion

A wide range of direct and indirect patient-related clinical outcomes have been reported in the literature and teams are encouraged to select and incorporate best-suited measures for their service delivery and development. We recommend regular monitoring and evaluation of clinical outcome data to benchmark services for acute older people. We also recommend the use of standardized and validated tools to measure indirect clinical outcomes including the caregiver burden and organizational-related outcomes. Clinical outcomes and dynamic data over time should be used to monitor the impact of QI initiatives on patient care. The methodology used to measure outcomes and QI initiatives should be shared and widely published. Clinical teams are encouraged to develop links with local research and development teams and work in collaboration with a university to build a research team, offering clinical research to the patients.

Acknowledgments

The authors are grateful to all members of the Department of Geriatric Medicine, Ysbyty Ystrad Fawr for their continued support for research activities. The authors would especially like to thank Rachel Fletcher, ABCi team, Aneurin Bevan University Health Board and Dr Chris Edwards, Research and Development, Aneurin Bevan University Health Board for their support and guidance. The authors would also like to thank Jane Power for her administrative support. The authors would also like to thank Salma Zabaneh for her contributions. No external funding was applied.

Author contributions

IS wrote the first draft and SA completed a critical revision of the review article. All authors contributed toward data analysis, drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

IS has attended the Research Methodology Course, Wales Deanery and the Introduction to Biostatistics course, Cardiff University. IS has also attended the Measurement Lead course from ABCi team, Aneurin Bevan University Health Board and completed Introduction to Good Clinical Practice eLearning (Secondary Care), National Institute for Health Research. IS has widely published service evaluation research and QI research, therefore learning from the listed courses and research experience has been used to draft this review article. The authors report no other conflicts in this work.

References

1.		Office of National Statistics. National Population 2012-based Statistical Bulletin. London: ONS; 2013. Available from: http://www.ons.gov.uk/ons/dcp171778_334975.pdf. Accessed April 10, 2018.
2.		Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380(9836):37–43.
3.		Conroy SP, Ansari K, Williams M, et al. A controlled evaluation of comprehensive geriatric assessment in the emergency department: the “Emergency Frailty Unit”. Age Ageing. 2014;43(1):109–114.
4.		Fox MT, Persaud M, Maimets I, et al. Effectiveness of acute geriatric unit care using acute care for elders components: a systematic review and meta-analysis. J Am Geriatr Soc. 2012;60(12):2237–2245.
5.		Sager MA, Franke T, Inouye SK, et al. Functional outcomes of acute medical illness and hospitalization in older persons. Arch Intern Med. 1996;156(6):645–652.
6.		Clegg A, Relton C, Young J, Witham M. Improving recruitment of older people to clinical trials: use of the cohort multiple randomised controlled trial design. Age Ageing. 2015;44(4):547–550.
7.		Barnes DE, Palmer RM, Kresevic DM, et al. Acute care for elders units produced shorter hospital stays at lower cost while maintaining patients’ functional status. Health Aff (Millwood). 2012;31(6):1227–1236.
8.		Blakemore S. Emergency frailty unit helps get patients home quicker. Nurs Older People. 2012;24(2):6–7.
9.		Aithal S, Patel P, Budihal D, Davies K, Ramakrishna S, Singh I. 131An Association Between Increasing Age And The Clinical Outcomes Of A Geriatrician-LED Emergency Frailty Unit (EFU) In An Enhanced Local General Hospital. Age Ageing. 2017;46(suppl_1):i35–i38.
10.		Hickman L, Newton P, Halcomb EJ, Chang E, Davidson P. Best practice interventions to improve the management of older people in acute care settings: a literature review. J Adv Nurs. 2007;60(2):113–126.
11.		Royal College of Physicians [webpage on the Internet]. Acute care toolkit 3: Acute medical care for frail older people. London: Royal College of Physicians; 2012. Available from: https://www.rcplondon.ac.uk/guidelines-policy/acute-care-toolkit-3-acute-medical-care-frail-older-people. Accessed April 18, 2018.
12.		David O. Base care on need, not age. BMJ. 2016;355:i5788.
13.		Simon Conroy; British Geriatrics Society. Silver Book. Quality care for older people with urgent & emergency care needs; 2012. Available from: http://www.bgs.org.uk/campaigns/silverb/silver_book_complete.pdf. Accessed April 2, 2018.
14.		Baztán JJ, Suárez-García FM, López-Arrieta J, et al. Effectiveness of acute geriatric units on functional decline, living at home, and case fatality among older patients admitted to hospital for acute medical disorders: meta-analysis. BMJ. 2009;338:b50.
15.		Beswick AD, Rees K, Dieppe P, et al. Complex interventions to improve physical function and maintain independent living in elderly people: a systematic review and meta-analysis. Lancet. 2008;371(9614):725–735.
16.		Ellis G, Whitehead MA, Robinson D, O’Neill D, Langhorne P. Comprehensive geriatric assessment for older people admitted hospitals: meta-analysis of randomised controlled trials. BMJ. 2011:343:d6553.
17.		Rubenstein LZ, Stuck AE, Siu AL, Wieland D. Impacts of geriatric evaluation and management programs on defined outcomes: overview of the evidence. J Am Geriatr Soc. 1991;39(9 Pt 2):8S–16S.
18.		Harari D, Martin FC, Buttery A, O’Neill S, Hopper A. The older persons’ assessment and liaison team “OPAL”: evaluation of comprehensive geriatric assessment in acute medical inpatients. Age Ageing. 2007;36(6):670–675.
19.		Charlson ME, Pompei P, Ales KL, Mackenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383.
20.		Fried LP, Tangen CM, Walston J, et al. Cardiovascular Health Study Collaborative Research Group. Frailty in older adults: evidence for a phenotype. Journal of Gerontology. Series A, Biological Sciences and Medical Sciences. 2001;56:146–156.
21.		Rockwood K, Mitnitski A. Frailty in relation to the accumulation of deficits. Journal of Gerontology. Series A, Biological Sciences and Medical Sciences. 2007;62:722–727.
22.		Searle SD, Mitnitski A, Gahbauer EA, Gill TM, Rockwood K. A standard procedure for creating a frailty index. BMC Geriatr. 2008;8:24.
23.		Singh I, Gallacher J, Davis K, Johansen A, Eeles E, Hubbard RE. Predictors of adverse outcomes on an acute geriatric rehabilitation ward. Age Ageing. 2012;41(2):242–246.
24.		Rockwood K, Song X, Macknight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ. 2005;173(5):489–495.
25.		Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index. Md State Med J. 1965;14:61–65.
26.		Singh I. Assessment and management of older people in the general hospital setting. Zawada Jr ET; 2016.
27.		Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9(3):179–186.
28.		Delgado-Hurtado JJ, Berger A, Bansal AB. Emergency department Modified Early Warning Score association with admission, admission disposition, mortality, and length of stay. J Community Hosp Intern Med Perspect. 2016;6(2):31456.
29.		Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P. The impact of hospital nursing characteristics on 30-day mortality. Nurs Res. 2005;54(2):74–84.
30.		Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. JAMA. 2003;290(12):1617–1623.
31.		Singh I. Training and Professional Development for Nurses and Healthcare Support Workers: Supporting Foundation for Quality and Good Practice for Care of the Acutely Ill Older Person. Int Arch Nurs Health Care. 2015;1(1):007.
32.		Singh I, Morgan K, Belludi G, Verma A, Aithal S. Does nurses’ education reduce their work-related stress in the care of older people? Journal of Clinical Gerontology and Geriatrics. 2015;6(1):34–37.
33.		Tourangeau AE, Doran DM, Mcgillis Hall L, et al. Impact of hospital nursing care on 30-day mortality for acute medical patients. J Adv Nurs. 2007;57(1):32–44.
34.		NHS Staff Survey 2017. NHS Survey Coordination centre: Englan; March 2018. Available from: http://www.nhsstaffsurveys.com/Caches/Files/P3088_ST17_National%20briefing_v5.0.pdf. Accessed April 15, 2018.
35.		Swiger PA, Patrician PA, Miltner RSS, Raju D, Breckenridge-Sproat S, Loan LA. The Practice Environment Scale of the Nursing Work Index: An updated review and recommendations for use. Int J Nurs Stud. 2017;74:76–84.
36.		Rigby H, Gubitz G, Phillips S. A systematic review of caregiver burden following stroke. Int J Stroke. 2009;4(4):285–292.
37.		George LK, Gwyther LP. Caregiver well-being: a multidimensional examination of family caregivers of demented adults. Gerontologist. 1986;26(3):253–259.
38.		Schulz R, Beach SR. Caregiving as a risk factor for mortality: the Caregiver Health Effects Study. JAMA. 1999;282(23):2215–2219.
39.		Kuzuya M, Enoki H, Hasegawa J, et al. Impact of Caregiver Burden on Adverse Health Outcomes in Community-Dwelling Dependent Older Care Recipients. The American Journal of Geriatric Psychiatry. 2011;19(4):382–391.
40.		Adelman RD, Tmanova LL, Delgado D, Dion S, Lachs MS. Caregiver burden: a clinical review. JAMA. 2014;311(10):1052–1060.
41.		Zarit SH, Reever KE, Bach-Peterson J. Relatives of the impaired elderly: correlates of feelings of burden. Gerontologist. 1980;20(6):649–655.
42.		Zarit SH, Orr NK, Zarit JM. The Hidden Victims Of Alzheimer’s Disease: Families Under Stress. New York: New York University Press; 1985.
43.		Bédard M, Molloy DW, Squire L, Dubois S, Lever JA, O’Donnell M. The Zarit Burden Interview: a new short version and screening version. Gerontologist. 2001;41(5):652–657.
44.		Novak M, Guest C. Application of a multidimensional caregiver burden inventory. Gerontologist. 1989;29(6):798–803.
45.		Graessel E, Berth H, Lichte T, Grau H. Subjective caregiver burden: validity of the 10-item short version of the Burden Scale for Family Caregivers BSFC-s. BMC Geriatr. 2014;14(14):23.
46.		Department of Health. High Quality Care For All. NHS Next Stage Review. London: DoH; 2008. Available from: https://www.kingsfund.org.uk/sites/default/files/briefing-high-quality-care-for-all-jo-maybin-ruth-thorlby-kings-fund-july-2008.pdf. Accessed April 12, 2018.
47.		Willke RJ, Burke LB, Erickson P. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials. 2004;25(6):535–552.
48.		Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
49.		EuroQol Group. EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208.
50.		Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30(6):473–483.
51.		Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21(2):271–292.
52.		Kingsley C, Patel S. Patient-reported outcome measures and patient-reported experience measures. BJA Educ. 2017;17(4):137–144.
53.		Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. 2015;350:g7818.
54.		Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13(1):211.
55.		Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–370.
56.		Greenberg SA. How to try this: the Geriatric Depression Scale: Short Form. Am J Nurs. 2007;107(10):60–69.
57.		Singh I, Edwards C, Okeke J. Impact of Cognitive Impairment on Inpatient Falls in Single Room Setting and its Adverse Outcomes. Journal of Gerontology and Geriatric Research. 2015;S4:S4-001.
58.		Knight S, Singh I. Profile of inpatient falls in patients with dementia: A prospective comparative study between 100% single rooms and traditional multibedded wards. Journal of Clinical Gerontology and Geriatrics. 2016;7(3):87–92.
59.		Singh I, Okeke J, Edwards C. Outcome of in-patient falls in hospitals with 100% single rooms and multi-bedded wards. Age Ageing. 2015;44(6):1032–1035.
60.		Singh I, Varanasi A, Williamson K. Assessment and management of dementia in the general hospital setting. Rev Clin Gerontol. 2014;24(03):205–218.
61.		Kuzuya M, Enoki H, Hasegawa J, et al. Impact of caregiver burden on adverse health outcomes in community-dwelling dependent older care recipients. Am J Geriatr Psychiatry. 2011;19(4):382–391.
62.		Whitley E, Ball J. Statistics review 1: presenting and summarising data. Crit Care. 2002;6(1):66–71.
63.		Whitley E, Ball J. Statistics review 2: samples and populations. Crit Care. 2002;6(2):143–148.
64.		Moran JL, Solomon PJ. Statistics in review Part I: graphics, data summary and linear models. Crit Care Resusc. 2007;9(1):81–90.
65.		Moran JL, Solomon PJ, Review Sin. Statistics in review. Part 2: generalised linear models, time-to-event and time-series analysis, evidence synthesis and clinical trials. Crit Care Resusc. 2007;9(2):187–197.
66.		Marusteri M, Bacarea V. Comparing groups for statistical differences: how to choose the right statistical test? Biochem Med. 2010;20(1):15–32.
67.		In J, Lee S. Statistical data presentation. Korean J Anesthesiol. 2017; 70(3):267–276.
68.		Wheeler, Donald J. and Chambers, David S. Understanding Statistical Process Control. Knoxville, TN: SPC Press; 1992.
69.		NHS Wales, UK. 2015. Welsh Government [webpage on the Internet]. Making Prudent Healthcare Happen – An Update. Welsh Heath Circular. Available from: http://www.prudenthealthcare.org.uk/principles/. Accessed March 2, 2017.
70.		Prochaska JO, Diclemente CC. Stages and processes of self-change of smoking: toward an integrative model of change. J Consult Clin Psychol. 1983;51(3):390–395.
71.		Prochaska JO, Diclemente CC, Norcross JC. In search of how people change. Applications to addictive behaviors. Am Psychol. 1992;47(9):1102–1114.
72.		NHS Institute for Innovation and Improvement. Improvement Leaders’ Guides. Series 1: Process mapping, analysis and redesign. London: Department of Health; 2005.
73.		Nelson E, Batalden P, Godfrey M. Quality by Design: A Clinical Microsystems Approach. San Francisco: Jossey-Bass; 2007.
74.		Colligan L, Anderson JE, Potts HW, Berman J. Does the process map influence the outcome of quality improvement work? A comparison of a sequential flow diagram and a hierarchical task analysis diagram. BMC Health Serv Res. 2010;10:7.
75.		NHS Improvement [webpage on the Internet]. Creating Driver Diagrams for Improvement Projects. Central London Community Healthcare NHS Trust, UK. Available from: https://improvement.nhs.uk/resources/creating-driver-diagrams-for-improvement-projects/. Accessed April 25, 2017.
76.		Revere L, Black K. Integrating Six Sigma with total quality management: a case example for measuring medication errors. J Healthc Manag. 2003;48(6):377–391.
77.		Young T, Brailsford S, Connell C, Davies R, Harper P, Klein JH. Using industrial processes to improve patient care. BMJ. 2004;328(7432):162–164.
78.		Derosier J, Stalhandske E, Bagian JP, Nudell T. Using health care Failure Mode and Effect Analysis: the VA National Center for Patient Safety’s prospective risk analysis system. Jt Comm J Qual Improv. 2002;28(5):248–267.
79.		Walley P, Gowland B. Completing the circle: from PD to PDSA. Int J Health Care Qual Assur. 2004;17(6):349–358.
80.		NHS Improvement [webpage on the Internet]. Plan, Do, Study, Act (PDSA) Cycles and the Model for Improvement. Central London Community Healthcare NHS Trust, UK. Updated January 17, 2018. Available from: https://improvement.nhs.uk/resources/pdsa-cycles/. Accessed April 4, 2018.
81.		Walley P, Gowland B. Completing the circle: from PD to PDSA. Int J Health Care Qual Assur Inc Leadersh Health Serv. 2004;17(6):349–358.
82.		Singh I, Okeke J. Reducing inpatient falls in a 100% single room elderly care environment: evaluation of the impact of a systematic nurse training programme on falls risk assessment (FRA). BMJ Qual Improv Rep. 2016;5(1):u210921.w4741.
83.		Singh I, Edwards C, Anwar A. One-Year Mortality Rates Before and After Implementing Quality-Improvement Initiatives to Prevent Inpatient Falls (2012–2016). Geriatrics. 2018;3(1):9.
84.		Aithal S, Sequeira R, Edwards C, Singh I. Fragility Fractures and Parkinsonism: Relationship of Fractures with Demography, Severity and Predictors of Adverse Outcomes. Geriatrics. 2017;2(2):17.
85.		Singh I, Fletcher R, Scanlon L, Tyler M, Aithal S. A quality improvement initiative on the management of osteoporosis in older people with Parkinsonism. BMJ Qual Improv Rep. 2016;5(1):u210921.w5756.
86.		Singh I, Anwar A, Rasuly A, et al. Prevalence of fragility fractures according to quality and outcomes framework (QOF) of the general medical services (GMS) contract and quality initiatives to improve osteoporosis care in the general practice within Caerphilly County Borough, Wales, UK: a feasibility study. MOJ Gerontology and Geriatrics. 2018;3(2):98–104.
87.		Segre LS, Buckwalter KC, Friedemann ML. Strategies to engage clinical staff in subject recruitment. J Res Nurs. 2011;16(4):321–332.