A Clinical Audit Of Service User And Carer Involvement In An NHS Community Mental Health Team Risk Assessment Tool

Introduction

National Health Service (NHS) England guidance suggests that allowing service users to be in control of their own care helps them to choose treatments that align with their preferences and goals.¹ This can consequently have a positive impact on their physical and mental health and well-being.¹ Individuals who struggle with their mental health may pose a risk to themselves (in the form of self-neglect, self-harm or suicide), to others (in the form of violence/neglect of others), or be at risk from others (exploitation/vulnerability). An individual who uses mental health services is referred to as a “service user”. Involvement in care and treatment for service users includes involvement in the assessment of these risks. United Kingdom (UK) Department of Health best practice guidelines in managing risk stipulate that risk management plans should be created on the basis of discussions between the service user and the staff that care for them.² Service user involvement in risk assessment is an important clinical process that has not been well researched; a recent systematic review that examined shared decision-making in risk assessment for violence found just five studies worldwide that met inclusion criteria.³

When service users have been interviewed about their involvement in risk assessment, most said that they were not told that risk assessments about them had been completed, or assumed that they had been without their involvement.⁴ Because service users may have appropriate self-knowledge to predict their own future attitudes and actions,⁵ involvement in risk assessment offers the benefit of self-prediction compared to prediction by others.^3,6 A recent national inquiry into suicide and safety within the UK indicated that service users and carers emphasised the importance of their involvement in risk assessment.⁷ Lack of service user or carer involvement could therefore exclude important and relevant information. Carers in particular have welcomed risk assessments that have been collaboratively created between service user, carer, and staff, that includes consideration of the family context.⁷ The exclusion of service user or carer involvement in risk assessment and management planning could therefore impact on their health, safety, and well-being and negate the possibility of a shared understanding of risk.

Identifiable NHS Trust and team information have been omitted from this paper because this audit forms part of the author’s assessment as a Trainee Clinical Psychologist, and is submitted to their university. This audit was conducted within an NHS Community Mental Health Team (CMHT) in the North-West of England, and is the first audit of service user involvement in risk assessment for this team. Community Mental Health Teams in England support people with challenging mental health needs in the community, rather than in an inpatient setting. Service users may come to a CMHT building, or mental health professionals may see service users in their own home or similar setting.

There is a wide variety of risk assessment tools used within UK mental health services.⁷ In the current audited CMHT, the tool used to assess and manage risk and plan interventions is the Standard Tool for Assessment of Risk version 2 (STAR v2). Staff complete this assessment on the host Trust’s electronic record system. The STAR v2 captures the service user’s presenting risk, as well as predisposing, precipitating, perpetuating, and protective factors linked to this presenting risk. The assessment allows for a narrative formulation of presenting risk to be captured, and includes information towards the end of the assessment about the service user’s involvement in the risk assessment process.

The aim of this audit was to ascertain how many service users (or carers of service users) over a 12-month period had been involved in their risk assessment, and to what extent their involvement had influenced risk management planning. “Service user and carer involvement” shall henceforth be referred to in this paper as “service user involvement”. Some service users had multiple STAR v2 assessments on the electronic record system. To examine the most clinically relevant information, only the service user’s most recent STAR v2 assessment was accessed and audited.

Service user involvement in risk assessment can take different forms.⁴ For the purposes of this audit, service user involvement was defined as positive indication in the STAR v2 that the service user was involved in the risk assessment; the service user agreed/disagreed with the assessment and; the service user noted their view of risk(s). The latter in practice includes contributing to the identification of risk factors or management plans.

Audit Standards

Service user involvement in risk assessment is included as a minimum standard for good practice within the host Trust’s internal Clinical Risk policy (reference omitted to maintain anonymity). The standards to be measured against were:

1. “Service users and carers (having regard to service user’s wishes) are involved”.
2. “Contingency/crisis plans are developed with the service user and (if appropriate) their family/carer”.

Given that these standards are considered minimum good practice, the benchmark to be measured against was 100%. If a clinically relevant reason had prevented involvement (e.g. it would have been unsafe to do so or the individual lacked capacity to participate), then this should have been clearly documented within the STAR v2.

During the set-up phase of this audit, several staff were asked for their thoughts on service user involvement in risk assessment. Anonymised feedback included:

"Client involvement allows for exploration of therapeutic risk taking and ownership of risk"

"The main thing I tend to encounter in this process is ambivalence towards collaborating in the risk assessments"

As part of their routine care, some service users within the CMHT were asked their thoughts on involvement in their risk assessments. Anonymised feedback included:

"I wasn’t offered at the start because I was too unwell, but I don’t really want to be involved as I’m used to others doing things for me"

"I don’t know what a risk assessment is"

Staff and service users consented to anonymised feedback being included in this paper. Because the current paper reflects an audit of routine clinical activity, approval for this audit was provided by the host Trust’s clinical audit department. Individual consent from audited service user records was not required.

Methods

The host Trust’s Business Intelligence Department provided data for 194 service users accepted into the CMHT from 31 October 2017 to 1 November 2018. Figure 1 details the data identification process.

Figure 1 Data identification process.

Of 189 accessible service user records, 83 were included in this audit (44%). The author has received comprehensive data confidentiality training, and was under the supervision of a senior member of staff throughout this process. Only the STAR v2 document was examined and accessed for each service user. The vast majority of most recent STAR v2 assessments had been completed by another team within the Trust (such as crisis or inpatient teams). In such cases, the service users’ STAR v2 was not accessed.

A bespoke audit tool was created for data collection, with five questions used to examine compliance with standards:

1. Has the service user been involved in the assessment process?
2. Is it indicated whether the service user agrees with the assessment outcomes?
3. Is there evidence of the service user or carer’s views?
4. Is there evidence of collaboration with the service user in identifying the service user’s own interventions or adaptive/maladaptive coping mechanisms?
5. If service user’s own interventions are present, are these evidenced in the risk formulation and management plan?

Questions one, two, and three on the audit tool related to and examined the audit standard “Service users and carers (having regard to service user’s wishes) are involved”. Forced choice yes/no/not applicable tick boxes for each of these questions on the STAR v2 were examined. If “not applicable” was ticked then this was marked as “no” for these questions, unless it was clearly described elsewhere in the assessment why this was not applicable.

Questions four and five on the audit tool related to and examined the audit standard “Contingency/crisis plans are developed with the service user and (if appropriate) their family/carer”. Free text boxes in the risk formulation and risk management sections of the STAR v2 were examined for evidence of involvement. For the purposes of standardization, statements including, but not limited to, “[service user] has suggested/understands that/agrees/thinks/recognizes/has told me/wishes to…” were considered to indicate collaboration. If collaboration was evident, then question five was scored as yes/no. If collaboration was neither confirmed nor disconfirmed, then question four was scored as “not applicable”.

On the bespoke audit tool, only the service user’s online record system identifier was included. This enabled a second member of CMHT staff, trained in clinical auditing, to complete a 10% data check to establish inter-rater consistency. There was 100% agreement between the author and independent staff member, indicating data were collected reliably and consistently. All data were shredded once results had been obtained.

Results

Audit results suggest that of the 83 participants, 70% were marked on their STAR v2 as involved in the assessment (Figure 2).

Figure 2 Overall involvement.

It was also clear that the majority of these service users had been asked whether they agree/disagree with the outcomes (70% of the sample; Figure 3).

Figure 3 Agreement with outcomes.

Evidence that service user/carer views were taken into account was apparent in only 36% of audited assessments (Figure 4).

Figure 4 Evidence of views.

Furthermore, only eight (10%) assessments clearly indicated collaboration between the service user and the staff member in identifying the service users’ own interventions or coping strategies (Figure 5).

Figure 5 Evidence of collaboration.

Of these eight assessments, only two indicated that the service users’ own interventions or strategies were then evidenced in the risk management plan. The vast majority of assessments (90%) did not evidence collaboration in identifying interventions/strategies (Figure 6).

Figure 6 Interventions into management plan.

Discussion

“Snapshotting” Involvement

It is important to consider the results of this audit within a clinical context. Doing so requires a reflection on the limitation of interpreting levels of involvement based on five questions included within a more comprehensive electronic document. It is entirely plausible that staff may have been clinically involving service users in risk conversations and assessment, and not recording their doing so. This raises the possibility that involvement has taken place more than is indicated by the current findings. The opposite to this could, of course, also be true, especially given that findings from this audit seem to correlate with the comments service users within the CMHT had made about their lack of involvement in the risk assessment process. One member of staff commented on service user ambivalence in collaborating with risk assessments and it is possible that staff have afforded opportunities for service users to be involved and these have been turned down. The exerting of control in choosing not to be involved may align with service user choice for their treatment;¹ recognition of the attempt to involve the service user should still be recorded on the STAR v2.

The involvement of service users in risk conversations may form part of other documents that staff in this particular CMHT have to complete as part of a service user’s care. This paper reports on an audit of the main risk assessment tool used within the CMHT; assessment of other relevant care documents may have indicated differing levels of involvement. The consideration of this poses two important reflections; if service users are being involved in other documents related to their care, and not their risk assessment, why is this so? Secondly, if other documents are recording involvement in risk-related conversations, why have these not made their way into the STAR v2 assessment? A “snapshot” of service user involvement over a 12-month period looking only at specific questions on a risk assessment tool may not be representative of actual involvement.

The STAR v2 As An Assessment Tool

Findings indicate that overall involvement and service user agreement/disagreement were markedly higher than service user/carer view of risk and intervention planning. This may be reflective of the way this information is recorded on the STAR v2. Overall involvement and agreement/disagreement with the outcome related to questions requiring a yes/no tick box response. Results indicated greater involvement in these areas. Lower levels of involvement (such as service users identifying their own interventions or management strategies) pertained to those questions with free text boxes that required staff to write in prose. When completing a lengthy form, the ticking of a yes/no box is arguably quicker and more convenient than typing free text. This is especially true if the staff member involved the service user by obtaining their views, however, the staff member can no longer recall the details of such discussions. The temptation to leave these boxes blank or omit service user involvement discussions may have appealed. It should therefore be considered that results could, in part, be reflective of the way this information is recorded on this specific risk assessment tool.

Procedural Challenges

When working in busy and demanding CMHTs, staff may come up against barriers or challenges that prevent adequate service user involvement in risk assessment. Procedural, systemic, or resource-related factors need to be considered when interpreting the findings of this audit. Procedurally, risk assessments often require renewal or updating after a certain period of time has elapsed. Staff renewing risk assessments may not have sought to involve the service user further if they were not involved previously, even though they may wish to be involved upon renewal. Previous findings that service users have often assumed that risk assessments have been completed without their involvement⁴ may offer an explanation for why service users felt prevented from actively voicing their desire to be involved. Similarly, if they were involved previously and there is no new risk information to record, there may have been a reluctance from staff to consult them again (even though they may have brought a unique risk perspective at that particular point in time).^5,6 Even if service users do not wish to be involved, it is worth considering the disappointment that carers have expressed at their lack of involvement in the risk assessment process.⁷ If service users do not wish to be involved, then consent could be sought from them to involve their carer instead.

Systemic Challenges

Systemically, it is recognized that mental health staff within this CMHT may have felt apprehensive or ill-equipped to have lengthy and detailed conversations of risk with service users they work with. Indeed, a national inquiry found that clinicians often lacked confidence in their ability to document meaningful risk information, and that risk assessment training was not updated frequently.⁷ In turn, this may lead to a reluctance to involve service users in risk assessment for fear of not feeling able to negotiate and manage the conversation. This may be a particularly important consideration for newly qualified staff, or staff who have not worked extensively in mental health care. In light of the current findings, this could be an important variable if service user involvement in other CMHT documents is being demonstrated.

Resource Challenges

From a resource perspective, results of this audit may reflect the lack of time and availability that staff have had to be able to fully involve service users and carers in their risk assessment. Staff may be under pressure to complete risk assessments within an allocated time frame that has prevented meaningful service user involvement. The administrative burden of completing risk assessments has previously been identified as an issue.⁷ If staff are recording risk-related information on other CMHT care documents, they may not deem it necessary, or have the time, to transfer this information into the STAR v2. Being mindful of procedural, systemic, or resource factors, it is possible that unknown variables have prevented involvement. For the current audit, this information was not collected or indicated on any audited STAR v2 assessments.

The Inability To Generalize

Importantly, the results of this audit have been obtained from just one NHS CMHT; generalizing to other CMHTs, NHS mental health services or non-NHS services is not possible. The examination of service user involvement in risk assessment in other CMHTs and mental health services would be beneficial. This may help clarify whether individual teams or services need to develop more successful ways of involving service users in risk conversations and assessment, or whether a broader and more pervasive omission of involvement in risk assessment exists within UK mental health services. It would furthermore be of interest to examine involvement in risk assessment in countries outside of the UK. Has the process of involving service users in risk assessment been successfully defined and practiced in other mental health systems? This would provide an excellent opportunity for learning and replication of processes that are both meaningful and efficient.

Conclusion

Service user involvement in risk assessment is essential, for reasons including the access to otherwise unknown information and a shared understanding of risk between service user and staff. Although the audited Trust’s standard for service user involvement in risk assessment was 100%, results of service user involvement, service user agreement, and service user views demonstrated this standard was not achieved. Limitations of the risk assessment tool have been considered, and the possibility of other variables impacting on a staff member’s ability to engage the service user has been discussed. Based on the results of this audit, recommendations are provided for general mental health services, to help facilitate future service user involvement in mental health risk assessment. These recommendations have been developed with the hope that they are applicable to both domestic and international mental health services.

Recommendations

1. Community Mental Health Teams and other mental health services (including inpatient, outpatient and day services within the UK, and other mental health services outside of the UK) should audit their levels of service user involvement in risk assessment.
2. If it is not already, then ways to meaningfully involve service users in risk assessment should be included in routinely delivered clinical risk training. It is of utmost importance that information aimed at helping staff negotiate risk conversations (including identifying interventions and navigating difficult subjects) with service users is included in such training. Having service users co-deliver this training should be considered, as this would provide staff with insight into what service users require when negotiating risk conversations (as has been described, “staff training on understanding risk and not just tool completion”).⁷ The desires and needs of service users in risk assessment may be very different to what mental health professionals expect. Finally, such training should include ways that staff can manage their own responses to the hearing of risk-related information, such as the use of clinical supervision or reflective spaces.
3. Staff should be afforded an opportunity to clarify barriers and challenges to involving service users in risk assessments, so blame is not unjustly attributed to those carrying out assessments. Focus groups could be a valuable method for facilitating these conversations. It would be useful if such discussions were facilitated by an individual independent to the staff team, so that open and honest reflections concerning the procedural, systemic, or resource-related barriers can be discussed.
4. If staff have clarified barriers or challenges to involvement, such barriers or challenges should be addressed by service management in order to maximize the potential for future service user involvement.