Validation of the distress thermometer for use among adolescents and young adults with cancer in Australia: a multicenter study protocol

Background

Each year in Australia, approximately 900 adolescents and young adults (AYAs) aged 15–24 years are diagnosed with cancer.¹ For AYAs, a cancer diagnosis coincides with the developmental transition from adolescence to young adulthood.² This critical life stage includes physical, psychological, and social developmental changes; major transitions from school to work or further education; and growing independence from families.^3,4 For AYAs diagnosed with cancer, the need to negotiate the dual demands of transitioning to adulthood, along with a cancer diagnosis and subsequent treatment, can lead to complex psychosocial needs and psychological distress, with correspondingly poorer psychosocial outcomes than those of patients from other age groups.⁵ Part of treating AYA cancer, therefore, necessarily involves developing appropriate psychosocial care plans, which in turn depends on the availability of screening tools that can effectively identify patients’ needs and elevated levels of distress.

Psychological distress

Distress is a broad psychological construct that incorporates clinical conditions such as depression and anxiety,⁶ as well as more specific concepts such as sadness, worry, and fear.⁷ The National Comprehensive Cancer Network (NCCN) in the US defines distress among cancer patients as being “… a multifactorial unpleasant emotional experience of a psychological (cognitive, behavioural, emotional), social, and/or spiritual nature that may interfere with the ability to cope effectively with cancer, its physical symptoms and its treatment”.⁸ The NCCN argued the case for distress to be classified as the sixth vital sign of cancer care, the first five being measurements of pulse, respiration, blood pressure, temperature, and pain.⁹ Canadian health care professionals have now endorsed this position,¹⁰ and work is underway to disseminate and implement the NCCN’s guidelines internationally.¹¹ The inclusion of distress in the list of vital signs highlights the importance of this construct and the need for a reliable means of identifying it.

The prevalence of elevated distress among AYAs diagnosed with cancer has been found to be higher than that among the general population, with figures in the vicinity of 40%,¹² compared to Australian norms for 16–24 year olds of 9% high or very high distress.¹³ Several variables have been associated with higher levels of patient distress, including: having not yet started treatment,^12,14 being diagnosed with leukemia,¹⁵ being a young adult compared with being an adolescent,^16,17 not being involved in school or work,¹⁴ having a less stable economic status,¹⁶ and having lower levels of support.¹⁷ Poor health literacy has also been associated with increased distress in people diagnosed with cancer.^18,19 There is also evidence of an association between increased patient spirituality and psychological well-being,^20,21 as well as perceived better family functioning and lower psychological distress in family members of childhood cancer survivors.^22–24 It should be noted that not all young people diagnosed with cancer demonstrate clinically significant distress and that those who do may first show symptoms at 6 or 12 months after diagnosis.²⁵ Accordingly, it is important that only those young people who would benefit from referral to psychosocial services be referred and that young people who are not in such need are not unnecessarily burdened.

Identifying distress – the distress thermometer

Numerous measures have been developed in order to ascertain patients’ levels of distress and determine whether referral to additional support is required.^6,26–32 For a measure to be useful, it needs to have good validity, be easy for the patient to use, and be easily interpreted by the health care professional. It is also important that there is high acceptability of a screening tool in order for health care professionals to feel confident about using it.³³

Despite the proliferation of distress measures in general, a recent systematic review found no validated measures of distress for AYAs diagnosed with cancer.³⁴ The review also found that taking into account the absence of any other measure suitable for AYAs with cancer, the distress thermometer (DT; described later) is potentially suitable for this purpose, a position seconded by the clinician discussion group involved in the development of the AYA Oncology Psychosocial Care Manual.^3,35 It has been suggested that the non-pathological term “distress” may be more acceptable to patients than terms such as “depression” and “anxiety”,^8,35,36 providing further grounds for adopting this approach in early screening procedures.

The DT is a single-item measure which consists of a “thermometer” with numerals displayed vertically from 0 to 10. Patients rate their distress “over the last week”, with 0 indicating “no distress” and 10 indicating “high distress”. Current NCCN guidelines suggest a cut-off of 4 (ie, scores of 4 or higher) for determining whether patients require referral to psychosocial services.⁸

Accompanying the DT is the problem checklist (PCL), which nominates specific categories and sub-items that could contribute to the patient’s levels of distress (eg, practical, family, emotional, social, physical, and information).^35,37 The PCL was originally developed for older adults, and therefore reflects their concerns. The aforementioned clinician discussion group and a working party of AYAs who had been diagnosed with cancer reviewed and modified the items in the PCL in order to create a more comprehensive picture of AYA patient need.³ Among the benefits of the combined DT/PCL are that it is quick and simple for patients to use and that it can be administered by members of the multidisciplinary team other than a psychologist, thereby increasing the likelihood that it will be used.³⁵

Although initial diagnosis and early treatment are evidently key times to be monitoring distress, other periods of transition (eg, relapse, treatment that coincides with major life events and decision reassessment) may be important as well.²⁵ Despite being a welcome milestone, the transition from active therapy into follow-up care (ie, survivorship) is a stressful time for AYA cancer patients, who are faced with the task of re-establishing a “normal” life, often without access to the supports that were available during the treatment period.^38–40 As such, attention has also been given to the development of a formalized AYA Oncology Psychosocial Survivorship Care Process,⁴¹ utilizing the DT and a PCL designed to capture potential areas of concern associated with psychosocial recovery and post-treatment functioning. As with the “on-treatment” version of the DT/PCL, the survivorship version is yet to be validated for use in AYAs with cancer.

Validating the distress thermometer and service responsiveness

Cutoff points and content validity

While the DT has been validated in numerous adult cancer populations across a range of countries,^{6,7,33,42–45} the two AYA versions of the DT/PCL (ie, on-treatment and survivorship) require validation to ensure that appropriate cut-off values are identified for this population to facilitate appropriate and timely referrals.

A meta-analysis of short screening tools for cancer-related distress pooled data from nine studies and across 1,477 adult cancer patients in order to test the diagnostic validity of the DT.³³ Combined pooled results were sensitivity =77%, specificity =66%, positive predictive value (ie, rate of detecting true positives) =56%, and negative predictive value (ie, rate of detecting true negatives) =84%. This means that “In the real world, of 100 people screened for broadly defined distress in cancer settings, the DT (at the cut-off suggested by the NCCN) would suggest 40 probable cases, of which 18 would be false positives, and 60 probable non-cases, of which 9 would actually be distressed and therefore missed”.³³ Accordingly, the DT demonstrates reasonable rule-out (“true negative”) results for distress, but less reliable rule-in (“true positive”) outcomes. The primary purpose of determining an appropriate cut-off for the DT will be to strike a balance between reliable judgments about whom to “rule in” and whom to “rule out” when screening for enhanced psychosocial care needs.

Clinical utility

The clinical utility of the modified DT/PCL is another important research consideration and will be examined from both patient and health care professional perspectives. Clinical utility may broadly be understood in terms not only of clinical and cost-effectiveness, but also “… practitioners’ perspectives about the usefulness, benefits, and drawbacks of an innovation for their working practice”.⁴⁶

To ensure comprehensive evaluation of the clinical utility of the DT/PCL, the present study will test for the tool’s clinical utility by using Smart’s multidimensional model.⁴⁶ Clinical utility is defined by the concepts of “appropriateness” (ie, effectiveness and relevance), “practicability” (ie, the relationship between the tool and the practitioner’s needs and capabilities), and “acceptability” (ie, patient and health care professional perspectives on ethics, psychological concerns, and service delivery). Smart’s model asserts that an overall assessment of clinical utility can be made by considering the degree to which a measure addresses the four individual concepts.⁴⁶ For patients, this will capture how practical it was to complete the measure and the usefulness of any subsequent referrals. For health care professionals, the emphasis will be on how the measure aided them in their clinical roles and the likelihood of their using the measure as standard practice.

The use of distress screening in developing care plans

Distress screening is the first part of the process intended to improve patient psychosocial outcomes.⁴⁷ However, the effectiveness of screening for distress is disputed when it is used independent of a structured referral process or the development of a care plan.⁴⁸ Additionally, questions have been raised about whether patients receive appropriate support and access to services without screening.⁴⁹ Distress screening programs require three components to be effective; namely, use of a screening tool, triage to services, and effective treatments. While it is beyond the scope of this study to examine directly the quality of support and referred services for AYAs, we will nonetheless assess service responsiveness by examining the number and type of referrals made, along with patients’ satisfaction with the referrals they received.

Prevalence and predictors of distress

This study will also assess the prevalence of distress among AYA on-treatment cancer patients and survivors, and identify potential predictors of distress, including patient demographics, health literacy, spirituality, family functioning, and perceived severity of illness.

Study aims

This study is designed to validate the DT/PCL for use with AYAs and to measure the tool’s acceptability and usability among both AYAs and health care professionals. The study will examine both the on-treatment and survivorship versions of the DT/PCL. The primary aims are to 1) determine appropriate cut-offs for clinical referral on the DT, 2) investigate the content validity of the modified PCL, and 3) assess the clinical utility of the tool from the perspective of both AYAs and health care professionals.

The secondary aims of the study are to 4) establish prevalence and predictors of distress in Australian AYA cancer patients (ie, demographics, cancer type, perceived severity of illness, health literacy, family functioning, and spirituality variables) and 5) examine the number and character (including uptake) of post-screening referrals made to psychosocial services.

Methods

AYA questionnaires

Time 1: on-treatment patients and survivors

Demographics and medical history

On-treatment patients and survivors will complete demographic questions, including sex, age, residential location, education, work and relationship status, living arrangements, and several cultural and linguistic diversity, and socioeconomic status (SES) measures. In addition, clinical variables will be collected, including cancer type, date of diagnosis, age at diagnosis, relapse status, treatment types, treatment purpose (ie, curative, palliative, or unsure), the date when treatment started, the date when treatment ended (for survivors), and treating hospital.

Health literacy: Functional, Communicative and Critical Health Literacy (FCCHL)

The FCCHL measure comprises 14 items which measure the abilities of understanding, communicating, and evaluating health information.⁵⁰ The items are structured as a sentence prompt and answer, and the respondent uses a 4-point Likert scale (“never” to “often”) to indicate the frequency they experience the content of each item. The scale consists of three domains: functional health literacy (“When reading things about cancer… the printing is too small to read”), communicative health literacy (“Since I was diagnosed with cancer … I get my information about cancer from lots of people and places”), and critical health literacy (“Since I was diagnosed with cancer … I check information to see if it is correct”). The FCCHL was originally developed for use with Japanese diabetic patients and was found to be a reliable measure with good internal consistency (Cronbach’s α=0.78), and good correlations with similar scales demonstrated construct validity. The scale has previously been adapted for use for AYAs with cancer with only minor changes to the scale’s factor structure.⁵¹

Spirituality: Functional Assessment of Chronic Illness Therapy – Spiritual Well-Being Scale (FACIT-Sp)

The FACIT-Sp was developed to measure spiritual well-being in people with cancer and comprises two subscales: sense of meaning and peace, and role of faith in illness. The two subscales combine to give a total score for spiritual well-being.⁵² The FACIT-Sp uses a 5-point Likert scale (“not at all” to “very much”) and has a score range of 0–48, with higher scores indicating greater spiritual well-being. The measure was initially validated on a sample of recently diagnosed (average of 29 months after diagnosis) cancer and HIV/AIDS patients in the US. The FACIT-Sp has good internal consistency (Cronbach’s α=0.87) and moderate-to-strong correlations with spirituality subscales of the Functional Assessment of Cancer Therapy – General, suggesting sound validity. For the purposes of the present study, we will only be administering the faith subscale, which contains four items and has a possible score range of 0–16. The faith subscale has shown equally high internal consistency (Cronbach’s α=0.88).

Family functioning: Family Relationship Index (FRI)

The FRI^22,53 measures family functioning and comprises 12 items, 4 from each of the 3 subscales taken from the Family Environment Scale.²² The three subscales assess family cohesion (eg, “There is a feeling of togetherness in our family”), expressiveness (eg, “We tell each other about our personal problems”), and conflict (eg, “Family members hardly ever lose their tempers”). The items are answered in a dichotomous “true” or “false” format. The possible range of scores is 0–12 with higher total scores indicating better family functioning. The measure has been validated in a sample of Australian families of cancer patients, demonstrating moderate reliability with internal consistency ranging from 0.48 to 0.70.²² The FRI’s test–retest reliability and construct validity have also been established for the original subscales of the Family Environment Scale.⁵⁴

Distress Thermometer and Problem Checklist

The DT/PCL is described earlier.

Hospital Anxiety and Depression Scale (HADS)

The HADS is a 14-item measure that gives scores for “possible” and “probable” anxiety and depression in individuals with physical health problems.⁵⁵ The measure consists of 14 items, 7 each for symptoms of anxiety (eg, “I feel tense or wound up”) and depression (eg, “I have lost interest in my appearance”), with responses tailored to each symptom in terms of either severity or frequency as applicable. While the HADS was originally validated for use with people aged 16–65 years,⁵⁵ it has more recently been validated for use with adolescents aged 12–17 years.³¹ The HADS excludes symptoms of anxiety or depression that could be a result of physical illnesses (eg, chronic tiredness) in order to more specifically measure mental illness, thereby making the tool useful for assessing distress in patients with chronic illnesses. In the original validation, internal consistency was judged by correlating each subscale item with the total score of the remaining items within the subscale. Correlations ranged from r=0.30 to 0.76 (all significant at P<0.02). Diagnostic utility in judging the severity of a patient’s disorder has been demonstrated with strong, significant correlations between subscale scores and psychiatric ratings from clinical interviews (depression, r=0.70, P<0.001; anxiety, r=0.74, P<0.001).⁵⁵

The validation of the HADS for adolescents recommended total cut-off scores of 7 for the depression subscale and 9 for the anxiety subscale (ie, a total scale score of 16) for clinical settings where the aim is to minimize false negatives, and a higher cutoff of 10 for the depression subscale and 12 for the anxiety subscale (ie, a total scale score of 22) in general contexts that aim to minimize false positives.³¹ Studies involving adults have used other combinations of HADS cutoffs depending on the purpose of the study.^56,57

Kessler-10 (K10)

The K10 is a 10-item measure of psychological distress originally developed in Australia for use in nonclinical populations.⁵⁸ The K10 uses the sentence stem “During the last 30 days, about how often did …” with sentence completers such as “… you feel depressed?” and “… you feel tired out for no good reason?” Participants respond on a 5-point scale ranging from 1 (“never”) to 5 (“all the time”). The possible range of scores is 10–50, with higher scores indicating more distress. The K10 has excellent internal consistency (Cronbach’s α=0.93), validity, and specificity, as it discriminates well between cases and non-cases of DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, 4th edition) disorders and correlates strongly with the Global Assessment of Functioning. The K10 is widely used in Australian health contexts with normative data available for young people aged 16–24 years;¹³ it has also been used previously in AYAs with cancer.⁵⁹ In the present study, the K10 will be used to make national comparisons between the study participants and members of the general population matched for age and sex.

Perceived severity of illness

Measures of participants’ levels of perceived severity of their cancer diagnosis are adapted from Witte’s scale of severity of threat,⁶⁰ which comprises three items: severity, seriousness, and significance. The scale uses a 7-point Likert scale from “strongly disagree” to “strongly agree”. Initial validation of the perceived severity subscale demonstrated excellent reliability (Cronbach’s α=0.90). In the present study, we will be testing the value of perceived severity of illness as a predictor of distress.

Clinical utility questions

Clinical utility questions are based on three of the four components in Smart’s previously described multidimensional model: appropriateness, practicability, and acceptability.⁴⁶ The fourth component, “accessibility”, will not be used in our clinical utility analysis because it is concerned with issues that are not of high priority to the validation study, such as procurement, new technologies, and management of suppliers.

The clinical utility questions for patients have been adapted from those used by Breen et al,⁶¹ who previously examined clinician endorsement of the DT.⁶² The questions were developed using a sample of adult cancer patients in rural Victoria, Australia, as part of a study to evaluate the utility of a supportive care resource for clinicians. As the questions were developed for the purposes of validating a framework of supportive care with similar aims as the DT/PCL, they are appropriate for measuring clinical utility in this study.

The clinical utility questions for health care professionals regarding distress screening in practice (see section on Health care professional questionnaire) have been adapted from Ristevksi et al.⁶³ This study looked at how acceptable use of the DT/PCL was for health care professionals and was based on a sample of cancer clinicians from a number of hospitals in rural Victoria.⁶³

The clinical utility questions are distributed across three questionnaires: patients at Time 1 (T1), patients at Time 2 (T2), and health care professionals, and are the same for both the on-treatment patient and survivor components of the study.

The clinical utility questions for patients at T1 are asked immediately following completion of each of the three distress screening tools (ie, DT/PCL, HADS, and K10) and include: questions relating to (i) the just-completed tool’s relevance, (ii) ease of understanding, and (iii) impact on communication with the health care team (utilizing a 5-point Likert response scale, ranging from “strongly agree” to “strongly disagree”) and questions asking whether (iv) the tool covered the participant’s areas of concern and (v) how it might be improved.

Time 2 (6–8 weeks after time 1): on-treatment patients and survivors

Demographics and medical history

This survey includes: questions used to identify the patient and capture any changes in living, work, or study arrangements since T1 data collection; questions about cancer diagnosis, treatment purpose and status; and a question asking whether anyone had spoken to the patient about the possible impact of cancer or its treatment on future fertility.

The clinical utility questions for patients at T2 include: questions about service responsiveness, including the usefulness of referrals and the impact of T1 screening on subsequent treatment. The first three items are adapted from Breen et al,⁶¹ while the subsequent four items were developed by the study research team. All questions use the Likert response scale from T1 with an additional option of NA for patients who did not receive referrals. There is also a question concerning the number of referrals received since completing the DT/PCL at T1.

Health care professionals questionnaire

This survey includes questions about the health care professional’s workplace role, training, and experience; questions about distress screening and psychosocial assessments in practice (eg, frequency of usage, implementation of cutoffs); and questions about potential barriers to screening for distress (eg, staff uncertainty about how to identify distress), using a 6-point Likert scale from “not at all a barrier” to “very much a barrier”. The questions on potential barriers were adapted from a study examining barriers for health care professionals in using distress management guidance including the distress thermometer,⁶⁴ as well as from clinician feedback. The clinical utility questions for health care professionals examine the acceptability of using the DT/PCL (eg, the DT/PCL was easy for me to interpret, the DT/PCL has improved patient care), on a 5-point Likert scale from “strongly agree” to “strongly disagree”.

Study design

The study has two research aims. These are 1) a prospective validation study to assess the validity of the DT and PCL for use with AYAs with cancer and 2) a cross-sectional descriptive study to assess the prevalence and predictors of distress. The study draws on two separate clinical groups – on-treatment patients and patients in survivorship phase. These two groups will be analyzed independently and not compared statistically. Information collected from the health care professionals is descriptive and will assist in determining the validity and clinical utility of the DT/PCL.

Participants

Recruitment

AYA cancer patients will be recruited through one of five lead jurisdictional Youth Cancer Services (YCSs) across Australia. Located in major cancer treatment centers, YCSs comprise multidisciplinary clinical teams responsible for the direct and shared care of young people aged 15–25 years.

Potential patient participants will be identified by a nominated YCS team member at each hospital. Upon identification, potential participants will be provided with participant information and consent forms. Any questions concerning the study will be answered at this time. Contact details for the research team will be provided on the participant information form, along with helpline numbers.

Patient eligibility

Consenting young people aged 15–25 years diagnosed with cancer (excluding low-stage melanoma) who are either on-treatment (ie, within 1 month of diagnosis and/or first treatment cycle) or in early survivorship (ie, within 12 weeks of completing treatment) will be eligible to participate. AYA patients who are treated medically for their cancer outside of a YCS center but access the YCS for psychosocial support will also be eligible to participate.

Health care professional eligibility

All health care professionals involved in the psychosocial care and distress screening of AYAs within the YCS are eligible to participate.

Consent

Informed consent is required from each young person enrolled in the study. Guided by the NSW Health Policy statement, “It is NSW Health Policy that if the patient is under the age of 14 years, the consent of the parent or guardian is necessary”,⁶⁵ the present study (which will be recruiting individuals aged 15 years and over) is not seeking parental consent, except in a couple of instances where ethics committees have required parental consent. We have ensured that all documentation which will be read by participants have a Flesch reading ease score above 60.0, typically indicating that it can be read easily by 13 to 15-year-olds. We will provide participants with details of the study to take away with them, including research team contact details. Participants can choose to share this information with significant others, including parents. Consent will cover completion of the additional questionnaires, and access by the research team to the DT/PCL, the additional questionnaires, and critical information from the patient’s medical record. Information from the medical record will only be accessed when data collected on the questionnaires are incomplete. Consent for this will be given separately to the main study, on the same form.

Data collection

Time 1

Consenting participants (on-treatment patients and survivors) will complete the T1 questionnaire pack (described earlier). A copy of the DT/PCL will be kept by the hospital. For those patients declining to participate in the study, the DT/PCL will be administered as per standard admission procedures but the results will be excluded from the study data. Rates of, and reasons for, non-participation will be recorded.

Time 2 (6–8 weeks post-T1)

All AYAs completing T1 data collection will be invited to take part in a follow-up interview 6–8 weeks later. The semistructured telephone interview will capture changes in demographic and clinical variables that have occurred since data collection at T1 (ie, changes in: cancer stage, living arrangements, employment status, etc) and address patients’ longer term satisfaction with the DT/PCL and any associated referrals. If patients have not yet completed the DT/PCL for a second time (ie, 6–8 weeks after T1 data collection), they will complete it at the time of the telephone interview. Rates and reasons for non-participation will be recorded.

It is possible that some T1 on-treatment patients will have progressed to early-stage survivorship by T2 data collection. In these instances, participants will be eligible to participate, first, as on-treatment patients and secondly, as survivors. Individual participant data will not be tracked from on-treatment to survivorship. The flow of patient participants is illustrated in Figure 1.

Figure 1 Participant flow for on-treatment and survivor AYAs. Abbreviations: AYAs, adolescents and young adults; T1, time 1; T2, time 2; YCS, Youth Cancer Service.

Data collection will continue until target participant numbers are obtained (see later text). Participant timelines and data collection points appear in Table 1.

Table 1 Overview of study measures Notes: aDuring recruitment for groups 1 and 2; bwithin 4 weeks (preferably 2 weeks) of diagnosis (or relapse) and/or first treatment cycle; c6–8 weeks after diagnosis or treatment starting; d4–8 weeks of treatment finishing; e14–20 weeks after treatment finishing. Abbreviations: DT, distress thermometer; HADS, Hospital Anxiety and Depression Scale; PCL, problem checklist; T1, time 1; T2, time 2.

Privacy

Young people will initially be identifiable because of the necessity of accessing medical records and contacting participants for follow-up (T2). Data will be de-identified prior to data entry, and identifying details will be stored separately. All reported findings will be de-identified, aggregated data.

Statistical methods

Sample size

Assuming a large sample and approximately normal distribution, a prevalence of distress of 0.25, sensitivity/specificity of 0.70 and precision of 0.075, the required sample is 323.⁶⁶ Approximately 750 people aged 15–24 years are diagnosed with cancer (excluding melanoma) and require hospitalization each year in Australia.⁶⁷ Current YCS data indicate that approximately 55% of AYAs diagnosed with cancer are treated through a YCS center, an annual total of around 413 (unpublished data). The proportion of diagnosed young people accessing YCS centers is also increasing, with the first half of 2014 seeing an increase of over 10% in new patients (newly diagnosed and/or relapsed) referred into the YCSs, compared with the last 6 months of 2013 (unpublished data).

Allowing for recruitment delays and non-participation, it is expected that 330 on-treatment and 330 survivorship patients will be recruited during the 1-year T1 data collection period (on-treatment: January 2015–December 2015; survivorship: June 2015–May 2016). The sample will reflect the proportion of young people accessing each YCS and will not be a convenience sample. Using current administration numbers, Table 2 shows the number of participants from each jurisdiction that we intend to recruit (on-treatment and survivorship).

Table 2 Jurisdiction recruitment numbers Abbreviations: NSW, New South Wales; ACT, Australian Capital Territory; VIC, Victoria; TAS, Tasmania; QLD, Queensland; SA, South Australia; NT, Northern Territory; WA, Western Australia.

The T2 samples will be 50–70 patients for on-treatment and 50–70 for survivorship. In order to investigate the representativeness of the T2 samples, characteristics will be compared with those at T1 and the impact of any differences will be subject to further analysis.

Statistical analysis

In order to validate the DT as an effective measure of distress for AYA populations, a reference standard is required. We will be using the HADS measure as our comparator. Receiver operator characteristics analysis involving the HADS and the DT score will be used to assess the validity of the DT and determine suitable cut-offs. Using a cut-off score of 15 on the HADS-total, sensitivity (true positive rate) and specificity (1−false positive rate) of each score in the DT range will be calculated and used to determine how well the DT score distinguishes those who are distressed. Our aim is to maximize sensitivity and specificity; a minimum of 0.70 for sensitivity and specificity is considered to be indicative of a valid diagnostic measure.¹² It is preferable to have one cut-off score for all age groups, and for the purposes of screening, it is preferable to minimize false negatives and maximize sensitivity.

Additional analyses will be completed to examine the prevalence of distress among AYAs with cancer and the impact of variables such as age, sex, SES, cultural and linguistical diversity, cancer type and severity, health literacy, family functioning, and spirituality on levels of distress. The additional analyses will entail examination of the prediction of the distress score using a range of variables (eg, age, sex, SES, and the others listed). To this end, we will conduct hierarchical multiple linear regression with all of these variables included as predictors to determine which of these variables have direct and/or indirect associations with distress.

Clinical utility, satisfaction, and service responsiveness will all be assessed primarily using descriptive statistics. The impact of variables such as jurisdiction, patient remoteness and age, and health care professional level of experience or training will be used to explore the data further. Responses to open-ended questions will be analyzed using content analysis.

Ethics

The study has received research ethics committee approval in New South Wales (South Eastern Sydney Local Health District Human Research Ethics Committee, HREC [human research ethics committee] reference 14/112), South Australia (Women’s and Children’s Health Network, Royal Adelaide Hospital, HREC reference 14/113), Northern Territory (HREC of the Northern Territory Department of Health and Menzies School of Health Research, HREC reference 2014–2295), Queensland (Children’s Health Queensland Hospital and Health Service HREC, HREC reference 14/QRCH/374), Victoria (Peter Mac HREC, HREC reference 14/178), Australian Capital Territory (ACT Health HREC, HREC reference 11.14.311), and Western Australia (Sir Charles Gairdner Group HREC, HREC reference 2015-048).

Discussion

This study will provide important validation information for screening distress among AYA cancer patients and survivors. The primary purposes of this study are to: identify cut-off scores on the DT that are relevant and appropriate for AYAs diagnosed with cancer, thereby negating the need to rely on data from older adult populations; assess the content validity of the modified PCL; and assess the clinical utility of the tool from the perspective of both AYA patients and health care professionals. The incorporation of clinical utility measures for both patients and health care professionals will help to identify barriers to use and also provide information on the clinical responsiveness of the current distress screening and management processes.

Additionally, the distress prevalence and predictor data collected will enable better understanding of the psychosocial support needs of AYAs with cancer, thus helping to ensure that optimal support is made available to them. Findings from the study will allow necessary improvements to the distress screening and management process for on-treatment patients and survivors to be undertaken and may result in the implementation of services or changes in models of care, thus ensuring that psychosocial care within the YCS continues to be driven by evidence-based practice. Data on the effectiveness of the screening and associated processes will also improve confidence among health care professionals in using the tools.³³

Few previous studies have examined clinical utility with the DT/PCL, in particular whether patients and health care professionals were satisfied with both the DT and PCL, the extent to which using the tool improved patient–clinician communication, impacts on referral processes, and patient and health care professional views about the benefits of repeated screening.^63,68 One study reported that health care professionals and patients were mostly happy with the screening tools and resulting referral processes and that patients in particular were willing to be involved in repeat screening. However, there were some challenges to the tool being used by health care professionals, particularly concerning their questions about the extent of the empirical evidence demonstrating “when, for what, and for whom”⁶³ screening is beneficial (issues that are addressed through prevalence and predictor studies). Other feasibility issues concerned the need to provide patient privacy during screening and considerations of time.⁶³ These studies also examined participants’ views on the appropriateness of items included in the PCL, but did not directly question whether additional items should be added. The present study will pose this question directly.

This study is limited in its capacity to assess long-term patient-reported outcomes associated with distress screening. That is, the study is not designed to assess whether utilization of distress screening reduces distress among patients. This limitation is unavoidable because distress screening is only the first part of a lengthy process that requires adequate referrals and quality support services, a process that is beyond the scope of this study to follow-up and analyze. Thus, it is outside the scope of the study to address issues raised previously about whether distress screening is necessary.⁴⁹ Instead, the study will assess service responsiveness and patients’ satisfaction with their referrals. Similarly, the present study will not be tracking changes in distress over time beyond the T1 to T2 data collection timespan. Both aspects of long-term follow-up are important and interesting avenues for future research.

The study is also limited by the use of the HADS as an alternative measure rather than a clinical interview to assess distress levels and establish cut-offs for the DT. The decision to use an alternate measure of distress rather than a clinical interview was based on the need to recruit a large number of AYAs who are geographically distributed over a very large area, and the issues of feasibility that would arise in conducting a clinical interview with each of these young people.

Recruitment for this study is designed to be as inclusive as possible, thus maximizing the generalizability of the findings. Similar studies are being conducted internationally, and attention has been paid to ensure that there is a consistent minimum data set and methodology. Recent research has demonstrated the benefits of developing a consistent international approach to screening in terms of increased potential for pooling data and making international prevalence comparisons.^69,70 The present DT/PCL validation study will therefore be a collaborative study involving research partners from Australia, the UK, the US and Canada.

Acknowledgments

This study is funded by the Australian Government Department of Health, the Australian Youth Cancer Services and CanTeen Australia. CW is supported by a Career Development Fellowship from the National Health and Medical Research Council of Australia (APP1067501) and an Early Career Development Fellowship from the Cancer Institute of NSW (ID: 11/ECF/3-43).

PP is the Research and Youth Cancer Services General Manager at CanTeen, the Australian organization for young people living with cancer. She is also an Associate Professor in the School of Nursing at The University of Sydney. PP is a practicing psychologist with experience and expertise working with young people.

FM is the Research Manager at CanTeen and Adjunct Lecturer in the School of Nursing at The University of Sydney. She has considerable expertise working in the field of young people impacted by cancer.

AA is the Lead Clinician for Youth Cancer Services in NSW and ACT and the Director of the Sydney Youth Cancer Service.

DC is a Research Officer at the Psycho-oncology Co-operative Research Group at The University of Sydney, where he works primarily as a statistician and psychometrician.

CW leads the Behavioural Sciences Unit at the Kids Cancer Centre, Sydney Children’s Hospital. The Behavioural Sciences Unit is proudly supported by the Kids with Cancer Foundation. She is also a Senior Lecturer in the discipline of pediatrics, School of Women’s and Children’s Health, University of NSW. She is a registered psychologist and has conducted research with families affected by cancer for over a decade.

KW is the Cancer Institute of NSW Chair of Cancer Nursing and Director of the Cancer Nursing Research Unit, The University of Sydney.

KT is the Program Director of the ONTrac at Peter Mac Victorian Adolescent and Young Adult Cancer Service and the Service Manager of the Victorian/Tasmanian Youth Cancer Service.

MO is a hematologist/pediatric, adolescent, and young adult oncologist working at the Women’s and Children’s Hospital (North Adelaide) and Royal Adelaide Hospital. He is the Lead Clinician of the South Australia/Northern Territory Youth Cancer Service.

Author contributions

All authors contributed to the conception and design of the study, the drafting and revising of the study protocol, the approval of the final manuscript, and are accountable for all aspects of the protocol.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Australian Institute of Health and Welfare. Cancer in Adolescents and Young Adults in Australia. Cancer series No 62 [Catalogue Number: CAN 59]. Canberra, Australia: AIHW; 2011.
2.	Palmer S, Thomas D. A Practice Framework for Working with 15–25 Year-Old Cancer Patients Treated Within the Adult Health Sector. Melbourne, Australia: Victorian Adolescent & Young Adult Cancer Service, Peter MacCallum Cancer Centre; 2008.
3.	Palmer S, Patterson P, Thompson K. A national approach to improving adolescent and young adult (AYA) oncology psychosocial care: the development of AYA-specific psychosocial assessment and care tools. Palliat Support Care. 2014;12(3):183–188.
4.	Patton GC, Viner R. Pubertal transitions in health. Lancet. 2007; 369(9567):1130–1139.
5.	Rosenberg A, Wolfe J. Palliative care for adolescents and young adults with cancer. Clin Oncol Adolesc Young Adults. 2013;3:41–48.
6.	Gessler S, Low J, Daniells E, et al. Screening for distress in cancer patients: is the distress thermometer a valid measure in the UK and does it measure change over time? A prospective validation study. Psychooncology. 2008;17(6):538–547.
7.	Patel SK, Mullins W, Turk A, Dekel N, Kinjo C, Sato JK. Distress screening, rater agreement and services in pediatric oncology. Psychooncology. 2011;20:1324–1333.
8.	National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines): Distress Management. Accessed August 24, 2012.
9.	Holland JC, Bultz B. The NCCN guideline for distress management: a case for making distress the 6th vital sign. J Natl Compr Canc Netw. 2007;5:3–7.
10.	Bultz BD, Groff SL, Fitch M, et al. Implementing screening for distress, the 6th vital sign: a Canadian strategy for changing practice. Psychooncology. 2011;20(5):463–469.
11.	Lazenby M. The international endorsement of US distress screening and psychosocial guidelines in oncology: a model for dissemination. J Natl Compr Canc Netw. 2014;12(2):221–227.
12.	Dyson GJ, Thompson K, Palmer S, Thomas DM, Schofield P. The relationship between unmet needs and distress amongst young people with cancer. Support Care Cancer. 2012;20:75–85.
13.	Australian Institute of Health and Welfare. Young Australians: Their Health and Wellbeing 2011. Canberra, Australia: AIHW; 2011.
14.	Kwak M, Zebrack BJ, Meeske KA, et al. Prevalence and predictors of post-traumatic stress symptoms in adolescent and young adult cancer survivors: a 1-year follow-up study. Psychooncology. 2013;22(8):1798–1806.
15.	Neville K. Psychological distress in adolescents with cancer. J Pediatr Nurs. 1996;11(4):243–251.
16.	Kim MA, Yi J. Psychological distress in adolescent and young adult survivors of childhood cancer in Korea. J Pediatr Oncol Nurs. 2013;30(2):99–108.
17.	Hatcher H, Lack C, Walsh C, et al. Psychological study of adolescent and young adult cancer patients and their parents throughout and beyond their cancer treatment: a pilot study. J Clin Oncol. 2012;30(Suppl):abstract 9589.
18.	Koay K, Schofield P, Gough K, et al. Suboptimal health literacy in patients with lung cancer or head and neck cancer. Support Care Cancer. 2013;21(8):2237–2245.
19.	Sharp LK, Zurawski JM, Roland PY, O’Toole C, Hines J. Health literacy, cervical cancer risk factors, and distress in low-income African-American women seeking colposcopy. Ethn Dis. 2002;12(4):541–546.
20.	Smith ED, Stefanek ME, Joseph MV, Verdieck MJ, Zabora JR, Fetting JH. Spiritual awareness, personal perspective on death, and psychosocial distress among cancer patients. J Psychosoc Oncol. 1994;11(3):89–103.
21.	Visser A, Garssen B, Vingerhoets A. Spirituality and well-being in cancer patients: a review. Psychooncology. 2010;19:565–572.
22.	Edwards B, Clarke V. The validity of the family relationships index as a screening tool for psychological risk in families of cancer patients. Psychooncology. 2005;14(7):546–554.
23.	Edwards B, Clarke V. The psychosocial impact of a cancer diagnosis on families: the influence of family functioning and patients’ illness characteristics on depression and anxiety. Psychooncology. 2004;13:562–576.
24.	Ozono S, Saeki T, Mantani T, et al. Psychological distress related to patterns of family functioning among Japanese childhood cancer survivors and their parents. Psychooncology. 2010;19(5):545–552.
25.	Zebrack BJ, Corbett V, Embry L, et al. Psychological distress and unsatisfied need for psychosocial support in adolescent and young adult cancer patients during the first year following diagnosis. Psychooncology. 2014;23(11):1267–1275.
26.	Chan YF, Leung DY, Fong DY, Leung CM, Lee AM. Psychometric evaluation of the Hospital Anxiety and Depression Scale in a large community sample of adolescents in Hong Kong. Qual Life Res. 2010;19:865–873.
27.	Furukawa TA, Kessler RC, Slade T, Andrews G. The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-being. Psychol Med. 2003;33:357–362.
28.	Gao W, Stark D, Bennett M, Siegert RJ, Murray S, Higginson I. Using the 12 item General Health Questionnaire to screen psychological distress from survivorship to end-of-life care: dimensionality and item quality. Psychooncology. 2011;21(9):954–961.
29.	O’Connor S, Carney T, House E, Ferguson E, Caldwell F, O’Connor R. Revision of the Hospital Anxiety and Depression Scale (HADS) to produce the Paediatric Index of Emotional Distress (PI-ED). PRO Newsl. 2010;43(Spring):2–4.
30.	Pietsch K, Allgaier A, Fruhe B, Hoyler A, Rohde S, Schulte-Korne G. Screening for adolescent depression in paediatric care: validity of a new brief version of the Center for Epidemiological Studies Depression Scale. Child Adolesc Mental Health. 2013;18:76–81.
31.	White D, Leach C, Sims R, Atkinson M, Cottrell D. Validation of the Hospital and Depression Scale for use with adolescents. Br J Psychiatry. 1999;175:452–454.
32.	Recklitis CJ, Parsons SK, Shih M-C, Mertens A. Factor structure of the Brief Symptom Inventory-18 in adult survivors of childhood cancer: results from the Childhood Cancer Survivor Study. Psychol Assess. 2006;18(1):22–32.
33.	Mitchell AJ. Pooled results from 38 analyses of the accuracy of distress thermometer and other ultra-short methods of detecting cancer-related mood disorders. J Clin Oncol. 2007;25:4670–4680.
34.	Wakefield CE, Patterson P, McDonald FEJ, Wilson HL, Davis E, Sansom-Daly UM. Assessment of psychosocial outcomes in adolescents and young adults with cancer: a systematic review of available instruments. Clin Oncol Adolesc Young Adults. 2013;3:13–27.
35.	CanTeen. Adolescent and Young Adult Oncology Psychosocial Care Manual. Sydney, Australia: CanTeen; 2011.
36.	Carlson LE, Waller A, Mitchell AJ. Screening for distress and unmet needs in patients with cancer: review and recommendations. J Clin Oncol. 2012;30(11):1160–1177.
37.	Youth Cancer Service SA/NT. Adolescent and Young Adult Oncology Psychosocial Survivorship Care Process. Adelaide, Australia: Youth Cancer Service SA/NT; 2012.
38.	Thompson K, Palmer S, Dyson G. Adolescents and young adults: issues in transition from active therapy into follow-up care. Eur J Oncol Nurs. 2009;13:207–212.
39.	Palmer S, Mitchell A, Thompson K, Sexton M. Unmet needs among adolescent cancer patients: a pilot study. Palliat Support Care. 2007;5(2):127–134.
40.	Wakefield CE, McLoone J, Butow P, Lenthen K, Cohn RJ. Support after the completion of cancer treatment: perspectives of Australian adolescents and their families. Eur J Cancer Care (Engl). 2013;22(4):530–539.
41.	Youth Cancer Service SA/NT (2012). Adolescent and Young Adult Oncology Psychosocial Survivorship Care Process. Adelaide: Youth Cancer Service SA/NT.
42.	Dolbeault S, Bredart A, Mignot V, et al. Screening for psychological distress in two French care centres: feasibility and performance of the adpated distress thermometer. Palliat Support Care. 2008;6(2):107–117.
43.	Gil F, Grassi L, Travado L, Tomamichel M, Gonzalez JR. Use of the distress and depression thermometers to measure psychosocial morbidity among Southern European cancer patients. Support Care Cancer. 2005;13:600–606.
44.	Hegel MT, Collins ED, Kearing S, Gillock KL, Moore CP, Ahles TA. Sensitivity and specificity of the distress thermometer for depression in newly diagnosed breast cancer patients. Psychooncology. 2008;15:556–560.
45.	Ransom S, Jacobsen PB, Booth-Jones M. Validation of the distress thermometer with bone marrow transplant patients. Psychooncology. 2006;15:604–612.
46.	Smart A. A multi-dimensional model of clinical utility. Int J Qual Health Care. 2006;18(5):377–382.
47.	Carlson LE. Screening alone is not enough: the importance of appropriate triage, referral, and evidence-based treatment of distress and common problems. J Clin Oncol. 2013;31(29):3616–3617.
48.	Hollingworth W, Metcalfe C, Mancero S, et al. Are needs assessments cost effective in reducing distress among patients with cancer? A randomized controlled trial using the distress thermometer and problem list. J Clin Oncol. 2013;31(29):3631–3639.
49.	Coyne JC. Benefits of screening cancer patients for distress still not demonstrated. Br J Cancer. 2013;108:736–737.
50.	Ishikawa H, Takeuchi T, Yano E. Measuring functional, communcative and critical health literacy among diabetic patients. Diabetes Care. 2008;31(5):874–879.
51.	McDonald FEJ, Patterson P, Costa DSJ, Shepherd HL. Validation of a health literacy measure for adolescents and young adults diagnosed with cancer. 2015.
52.	Peterman AH, Fitchett G, Brady MJ, Pharm LH, Cella D. Measuring spiritual well-being in people with cancer: the functional assessment of Chronic Illness Therapy – Spiritual Well-Being Scale (FACIT-Sp). Ann Behav Med. 2002;24(1):49–58.
53.	Kissane DW, Bloch S. Family Focused Grief Therapy. Berkshire, England: Open University Press; 2002.
54.	Moos RH. Conceptual and empirical approaches to developing family-based assessment procedures: resolving the case of the Family Environment Scale. Fam Process. 1990;29(2):199–208.
55.	Zigmond A, Snaith R. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67:361–370.
56.	Boyes A, D’Este C, Carey M, Lecathelinais C, Girgis A. How does the distress thermometer compare to the Hospital Anxiety and Depression Scale for detecting possible cases of psychological morbidity among cancer survivors? Support Care Cancer. 2013;21(1):119–127.
57.	Shim E-J, Shin Y-W, Jeon HJ, Hahm B-J. Distress and its correlates in Korean cancer patients: pilot use of the distress thermometer and the problem list. Psychooncology. 2008;17(6):548–555.
58.	Kessler RC, Andrews G, Colpe LJ, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med. 2002;32:959–976.
59.	Patterson P, McDonald FE. ‘Being mindful’: does it help adolescents and young adults who have completed cancer treatment? J Pediatr Oncol Nurs. Epub January 9, 2015.
60.	Witte K, Cameron KA, McKeon JK, Berkowitz JM. Predicting risk behaviors: development and validation of a diagnostic scale. J Health Commun. 1996;1(4):317–342.
61.	Breen S, Ristevski E, Regan M. Enabling supportive care screening and evidence-based referrals for patients with cancer: patient acceptability and clinician implementation of the Supportive Care Resource Kit (SCRK). Aust J Cancer Nurs. 2012;13(1):20–31.
62.	Ristevski E, Breen S, Regan M. Incorporating supportive care into routine cancer care: the benefits and challenges to clinicians’ practice. Oncol Nurs Forum. 2011;38(3):E204–E211.
63.	Ristevski E, Regan M, Jones R, Breen S, Batson A, McGrail MR. Cancer patient and clinician acceptability and feasibility of a supportive care screening and referral process. Health Expect. 2015;18(3):406–418.
64.	Tavernier SS, Beck SL, Dudley WN. Diffusion of a Distress Management Guideline into practice. Psychooncology. 2013;22:2332–2338.
65.	NSW Law Reform Commission. Young People and Consent to Health Care [Report 119]. Sydney, Australia: NSW Law Reform Commission; 2008.
66.	Kumar R, Indrayan A. Receiver Operating Characteristic (ROC) curve for medical researchers. Indian Pediatrics. 2011;48:277–287.
67.	Australian Institute of Health and Welfare A. Australian Cancer Incidence and Mortality Books. Canberra, Australia: AIHW; 2011.
68.	Fulcher CD, Gosselin-Acomb TK. Distress assessment: practice change through guideline implementation. Clin J Oncol Nurs. 2007;11(6):817–821.
69.	Mühlan H, Bullinger M, Power M, Schmidt S. Short forms of subjective quality of life assessments from cross-cultural studies for use in surveys with different populations. Clin Psychol Psychother. 2008;15(3):142–153.
70.	Mitchell AJ, Meader N, Davies E, et al. Meta-analysis of screening and case finding tools for depression in cancer: evidence based recommendations for clinical practice on behalf of the Depression in Cancer Care consensus group. J Affect Disord. 2012;140(2):149–160.