The feasibility of using electronic clinical outcome assessments in people with schizophrenia and their informal caregivers

Introduction

Clinical outcome assessments (COAs) measure a patient’s symptoms, overall mental state, or the effects of a disease or condition on how a patient functions and can be used to determine whether or not a health intervention has demonstrated a benefit (or otherwise) to recipients. This umbrella term encompasses many types of assessments including patient-reported outcomes (PROs), observer-reported outcomes (ObsROs), clinician-reported outcomes (ClinROs), and performance-related outcomes.

ClinROs and PROs are commonly used in schizophrenia to facilitate clinical diagnosis and evaluation of treatment benefit. There are a number of symptoms and impacts of schizophrenia which cannot be reliably reported by people with schizophrenia themselves, but may also not be observable to clinicians. Observer-reported outcomes are therefore often used to provide unique insights into observable signs and symptoms and levels of functioning in people with schizophrenia. Furthermore, the impact that caring for a person with schizophrenia can have on informal caregivers themselves is increasingly being recognized¹ and assessed as a means to understand the wider burden of disease.

Traditionally, COAs have been developed and validated using pen and paper. However, as digital technology has evolved, the use of electronic COAs (eCOAs) has become more prevalent. The use of eCOAs is associated with a number of advantages over pen-and-paper methods, particularly in terms of reducing missing data, providing time-stamped records, minimizing administrative burden, and avoiding secondary data entry errors.² When migrating an instrument from a paper to an electronic mode of administration, the resulting eCOA ought to produce data that are equivalent or superior to the data produced from the original paper version and this equivalency should be solely due to the change in mode.² In addition, it is essential to demonstrate the “usability” of the eCOA (ie, that respondents are able to use the software and the device appropriately).²

Although eCOA has been implemented with success in many populations, there is limited evidence regarding the feasibility of using eCOAs with people with schizophrenia. It has been suggested that people with schizophrenia are less “digitally proficient” than healthy controls^3,4 and are less likely to own a mobile device than the general population or people with other types of mental disorders.⁵ In addition, lower socioeconomic status⁶ and cognitive difficulties⁷ observed in schizophrenia populations are factors associated with less frequent use of computers.^8,9 Importantly, the issues encountered in this population are likely to differ according to the type of electronic device (eg, mobile versus on-site assessments), study design (eg, frequency of assessments), and the content of the instruments used (eg, recall of past week versus in-the-moment ratings).

The aims of the present study were to provide a comprehensive assessment of the feasibility of people with schizophrenia completing a range of validated PROs using a tablet computer in the context of an on-site clinical visit. This is in contrast to existing research which has centered predominately upon ecological momentary assessment using mobile technologies.^10–12 In addition, the present study also explores usability of eCOA devices for completion of a range of measures by caregivers of people with schizophrenia. Lastly, the ability of research support staff members to instruct and support participants when using an eCOA device is also explored.

Methods

Usability and cognitive debriefing interviews were performed with 12 people with schizophrenia, their 12 informal caregivers, and six members of research support staff. An informal caregiver was defined as “A person who has significant responsibility for managing the well-being of a person diagnosed with schizophrenia in an unpaid capacity”.¹³ Research support staff were defined as staff members within a physician’s office who would frequently interact with and administer questionnaires to patients.

A range of disease-specific and generic self-report instruments were debriefed among participants. Specifically, people with schizophrenia completed the following instruments on the electronic device:

• Schizophrenia Quality of Life Scale (SQLS).¹⁴ A 30-item disease-specific instrument assessing the impact of a person with schizophrenia’s symptoms on their quality of life.
• EuroQOL 5 Dimension (EQ-5D).¹⁵ A generic instrument comprising five items assessing mobility, self-care, usual activities, pain/discomfort, and anxiety/depression and one global item assessing the respondent’s perception of their health that day on a 0–100 visual analog scale (VAS).
• Short-Form 36 ([SF-36] v 2.0).¹⁶ A generic 36-item instrument assessing general health status which comprises eight domains that broadly assess physical and mental functioning.

Caregivers completed three instruments on the device:

• Schizophrenia Caregiver Questionnaire (SCQ).^1,17 A 32-item disease-specific instrument assessing the impacts of caring for a person with schizophrenia on the informal caregiver’s quality of life.
• Caregiver Global Impression scales (CaGI).^1,17 Three single-item measures to assess severity of the person with schizophrenia’s symptoms, change in the person with schizophrenia’s symptoms, and change in the experience of caregiving on a six- or seven-point Likert scale.
• SF-36 (v 2.0).¹⁶ As described above.

Instruments appeared on the device as one item per screen, with the exception of the EQ-5D, which had all five items on one screen and the VAS on a separate screen.

The changes from paper to electronic versions of the instruments used in this study were non-substantive (eg, small changes in instructions, changing to one item per screen) and no changes were made to item content or meaning. This level of modification is classed as minimal according to International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force guidelines and US Food and Drug Administration (FDA) guidance,^2,18 and, accordingly, cognitive and usability testing interviews are considered appropriate for providing evidence of equivalence.

Ethics

The study was conducted in accordance with the Declaration of Helsinki and was approved by Copernicus Group, a centralized independent review board (IRB) in the US. (IRB number: ADE2-12-453). Study procedures ensured that written informed consent was obtained from people with schizophrenia, their caregivers, and the members of research support staff prior to the collection of any data. All participants received financial compensation for the time spent participating in this study.

Recruitment

Twelve people with schizophrenia and their 12 informal caregivers were recruited into the study via referrals from private physicians. In addition, six members of research support staff were recruited. The sample size is deemed adequate given that ISPOR electronic patient reported outcomes (ePRO) Task Force guidelines recommend five to ten participants for studies of this type.² Participants were recruited from three sites in the US to avoid geographical bias: Baltimore, MD (eastern USA), St Louis, MO (central USA), and New Orleans, LA (southern USA).^19,20 To be included in the study, participants had to meet a number of predefined inclusion and exclusion criteria (Table 1). A series of quotas were employed to ensure a representative sample (Table 1).

Table 1 Study recruitment strategy Abbreviations: CGI, Clinical Global Impression; CGI-S, Clinical Global Impression of Severity; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; DSM-IV-TR, DSM-IV Text Revision; eCOA, electronic clinical outcome assessment.

eCOA device

The eCOA device used in this study (SITEpro^®; eResearch Technology Inc. Philadelphia, PA, USA) is a tablet with a 30.7 cm screen (Motion Computing J3500 Tablet PC with a Windows^® 7 Professional Operating System). The device weighed approximately 900 g and moved between screens at a maximum of 200 ms. Text was 16 point throughout the device. The devices were locked to functionalities specific to this study in order to simplify their use. Tablet devices are more portable than a traditional desktop computer and lighter and easier to hold than a laptop computer. Such devices also have fewer functional limitations in terms of the display (font size and VAS selection) than alternative mobile devices such as a personal digital assistant or smartphone.²

Interview procedure

All interviews were conducted by a trained interviewer using a semi-structured interview guide. Prior to the conduct of interviews, study staff members were asked to prepare the device for use, following instructions in a study-specific user manual. Staff members also received training on how to assist people with schizophrenia and their caregivers to use the device. People with schizophrenia and their informal caregivers were then interviewed concurrently by two interviewers. In the event that participants encountered difficulties, staff members/interviewers were available to assist the person with schizophrenia or caregiver in completing the instruments using the electronic device (but as the instruments are developed for self-completion by the respondent, staff members/interviewers were not to provide interpretation of instrument content).

Once the instruments had been completed, participants took part in a semi-structured interview to assess usability. Questions were initially broad (eg, “What did you think about using the eCOA device?”) and became more focused to ensure that all areas of interest were covered (eg, “What did you think of the layout of the questions on the device?”). Participants were also debriefed about any particular items they found difficult to answer and about specific aspects of the instruments which were modified in migration to electronic form. Once each staff member had assisted one person with schizophrenia and one caregiver in completing the instruments, they were also interviewed about their opinion of the ease with which people with schizophrenia and their caregivers were able to complete measures using the eCOA device and their experience of assisting participants in doing so.

Analysis

All interviews were audiotaped and transcribed verbatim for the purpose of qualitative analysis. Written interview transcripts were then entered into a qualitative software package (Atlas.Ti), which was used to facilitate the analysis of interview transcripts. Codes were assigned to statements which were then categorized into relevant domains. Analysis was conducted according to participant group, ie, people with schizophrenia, caregivers, and support staff.

Results

Demographic and clinical characteristics

The average age of people with schizophrenia recruited to take part in the study was 47 years, and males and females were equally represented (n=6). In accordance with recruitment quotas, people with a range of schizophrenia subtypes and severities were recruited (Table 2). The majority of caregivers were female, but male caregivers were adequately represented in the study sample (25%), and the majority of caregivers (83%) reported living with the person with schizophrenia. Caregivers reported having spent an average of 14 years caring for the person with schizophrenia and spending, on average, 47 hours per week providing such care. All but one of the support staff members reported having prior experience of administering questionnaires to patients in electronic form.

Table 2 Demographic and clinical characteristics Abbreviations: CGI-S, Clinical Global Impression of Severity; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; DSM-IV-TR, DSM-IV Text Revision; N/A, not applicable or not asked of that population; GED, General Educational Development.

People with schizophrenia

Device use

All 12 of the people with schizophrenia reported a positive experience of completing the instruments on the eCOA device, with nine (75%) spontaneously commenting that it was easy to use and four (33.3%) stating that it was easier than using pen and paper despite some patients having little experience using computers, smartphones, and tablets. When probed, all 12 of the people with schizophrenia said that they found it easy to use the eCOA device. Seven of the people with schizophrenia specifically commented that they liked the size of the device as it made the text easy to read. However, two people did state that the device was too big and could be made smaller. All of the people with schizophrenia said that they found it easy to use the touchscreen to select answers and three people specifically commented that the ability to see/hear what they had selected on the screen was useful. The majority of people (n=10; 83.3%) said that the device always responded to their touch first time, which was consistent with interviewer observations. When asked, most people liked the speed at which the device responded, although one person said that the device responded to their inputs “too fast”. Table 3 presents the participants’ quotes.

Table 3 Participant comments on electronic clinical outcome assessment completion Notes: Subject IDs are provided in parenthesis after quotes. IDs refer to site, subject type, gender, age and DSM-IV-TR subtype. Abbreviation: DSM-IV-TR, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition.

Instrument completion

The majority of issues to arise during instrument completion related to the content of the instruments rather than the electronic format per se. However, although there were no issues with understanding of the content of the EQ-5D, there were some issues specific to the electronic format of the EQ-5D in this study. The EQ-5D items were all displayed on one screen, and three of the 12 people with schizophrenia needed to confirm with the staff member/interviewer how the items were separated. In addition, five of the 12 people with schizophrenia had trouble selecting their response on the 0–100 VAS, either with the stylus or their finger. This was most likely a result of the narrow width of the scale. There were no trends with regard to age, schizophrenia subtype, computer familiarity, or issues making selections on the device.

Responses to the item layout were generally positive (n=11) and the majority of people found the text easy to read (n=10), although three participants suggested either making the text bigger or providing an option to make the text bigger. Most had no problem with the time it took to complete the instruments, even though they were completing multiple instruments using a think-aloud technique, thus increasing completion time. Only one person with schizophrenia said it took a long time to complete the instruments and, interestingly, this person was observed to complete the instruments in a shorter amount of time than others. Only two of the people with schizophrenia needed to go back and change their answers within an instrument. Reasons for this included confusion over what the item was asking or uncertainty over their chosen response.

Generally, individual items were presented on one screen, unlike a paper instrument, which would contain all items and thereby make it possible for participants to read all the items before responding. Five of the 12 people with schizophrenia said that they would have answered differently if they had been able to see all of the items at once; however, all who were asked said they would prefer to see the items individually on separate screens. Nine people with schizophrenia said that they found it easy to move onto the next instrument; however, some people suggested the device should show more clearly which instruments had been completed. When asked, all of the people with schizophrenia said that they would be willing to complete instruments on an eCOA device every 3 months.

Table 3 presents the participants’ quotes.

Caregivers

Caregivers of people with schizophrenia experienced few difficulties completing instruments on the eCOA device. Caregivers had a varied level of experience using computers, smartphones, or tablets and ranging from never having used a computer to using one every day. All caregivers reported finding the device easy to use, with seven saying that they preferred using an eCOA device to pen and paper, and some caregivers also said that they liked the speed and size of the device. However, device-specific issues mentioned by caregivers were that it was too heavy or bulky (n=4), and that the speed of response was slow (n=3). The only issue caregivers had with instrument completion was a dislike of repetitive statements (n=3). For example, “The following question is about activities you might do during a typical day” which appears for every question in the electronic format of the SF-36 but just once in the paper format, and “Over the past four weeks” which appears in every question in both paper and electronic formats of the SCQ. For the SF-36, this instruction appears for every question on the electronic version but once in paper format. There did not appear to be any differences in the issues experienced based on the age or computer familiarity of caregivers. Table 3 presents the caregivers’ quotes.

Staff members

All staff members used computers daily at work; however, some had little experience of using smartphones or tablets. Generally, staff members found the device easy to set up and to use. The issues staff members experienced with the device itself were associated with the need to press an arrow symbol to move onto the next question (n=1) and the use of a stylus to select answers (n=1), the size of the device (n=2), the orientation of the screen (n=1), the speed of response (n=1), and the size of the text (n=1). None of these issues, however, were a barrier to preparing the device for use by study participants.

Staff members also found it easy to guide people with schizophrenia and their caregivers on how to complete instruments on the device. They reported that people with schizophrenia and their caregivers found it easy to use the device itself; only one staff member reported that some people with schizophrenia found it hard to move onto the next question using the arrow. Staff member suggestions for improvements to the device included an option to increase text size and putting the device on a stand to improve readability.

There were some concerns from staff members that people with schizophrenia were not using the correct recall period but were thinking back over a longer period of time, and this was also noted by interviewers. Specifically, only one person with schizophrenia used the correct recall period when completing the SQLS, and less than half of the caregivers used the correct recall period when completing the SCQ. This issue is not specific to an electronic mode of administration, and participant understanding and use of the recall period for each instrument, in both paper and electronic format, could be further investigated.

Table 3 presents the staff members’ quotes.

Discussion

Although there is evidence of the use of electronic data capture on mobile devices (eg, ecological momentary assessment) among people with schizophrenia,^10–12,21 the feasibility of administrating eCOAs on-site in tablet mode has not been assessed. For many clinical research studies, collection direct from patients and their caregivers during scheduled site visits presents the most viable option of data collection (practical reasons, ensuring standardization of timing of assessments, etc). It is important, therefore, to assess the feasibility of data collection (and specifically the usability of electronic devices) used to collect data in such instances.

Findings from the current study provide evidence that people with schizophrenia and their caregivers are able to complete COA instruments using an electronic device, with many preferring this mode of completion to traditional pen-and-paper methods. These findings support those from previous studies where no differences in ease of device use between people with schizophrenia and healthy controls were observed.^21,22 That there was no difference in ease of completion of eCOAs according to participants’ familiarity of using computers or similar electronic devices is also consistent with previous findings.^12,22 Furthermore, no differences in ease of completion according to respondent age, symptom severity, and education were observed. Despite the small sample size included in this study, the finding that variables such as age, computer familiarity, and symptom severity did not affect ease of device use or eCOA completion may indicate that on-site eCOA could also be a feasible method of data collection in other mental illnesses or elderly populations (subject to further research).

Observations from this study highlight several considerations for future use of eCOAs. Although only a small number of difficulties were reported by participants, findings from the study highlight the importance of having a staff member available nearby to assist and resolve any issues encountered (specific to the electronic platform) if the eCOA is being completed in a clinical trial setting.

Repeated assessments are a requirement for all interventional studies that seek to demonstrate change over time, and there is a balancing act between the collection of information from participants and responder burden. In this study, participants indicated that they would not find completing repeated eCOAs over a period of time overly burdensome, suggesting that this should perhaps be less of a concern. This is supported by a study by Arbuckle et al, which demonstrated that study participants completing eCOAs at multiple time points could do so quickly and did not find it overly burdensome, suggesting concerns over respondent burden may often be unfounded.²³

Participants preferred to see individual items on separate screens, ameliorating concerns raised by Gwaltney et al⁴ that having one item per screen removes the option for participants to review all of their answers. Participants struggled to select an answer on the EQ-5D VAS, likely because of the width of the scale on the device screen, and despite the selected value being displayed on the device, difficulties selecting an answer could lead to responses that are not reflective of true health status. The EQ-5D is a widely used measure, and relatively small between and within participant differences on the VAS (7–10 points) have been identified as minimal important differences in other populations.²⁴ This is an important consideration, especially when participants are completing measures off-site (eg, at home) rather than on-site and help is less accessible.

Findings from the study suggest that repetition of recall period on each screen does not eliminate incorrect recall by participants. Based on these findings alone, it is difficult to determine whether this applies to all modes of administration or is specific to eCOA. To some extent, the design of the current study (ie, in-depth debriefing of measures following completion at a single time point in a research facility) may have influenced findings. However, incorrect recall has previously been demonstrated in both paper and electronically administered COAs.²⁵ Regardless of mode of administration, incorrect use of recall presents challenges to evaluating changes on respondent-reported parameters over time and is something to consider when using instruments in this manner.

Feedback regarding the usability of eCOA was consistent among people with schizophrenia and their caregivers. Accordingly, none of the issues encountered in this study suggest that people with schizophrenia are any less able to use eCOA than healthy individuals; however, further investigation is required in this area. It is noted that the sample in this study is limited to the US and does not include people with residual schizophrenia or people with schizophrenia who cannot identify a formal caregiver. A conceivable limitation of the study is therefore that the findings may differ geographically and in people with residual schizophrenia or with no informal caregiver. This could be a topic for further investigation. Modern technology provides endless opportunities for real-world and real-time data capture and future research could investigate the feasibility of utilizing more advanced mobile technologies (eg, participants’ own personal tablets and smartphones) to collect COA data in people with schizophrenia and their caregivers on devices that are already familiar to them. Other uses of electronic devices (eg, assessing adherence and compliance to treatment, patient education, and tracking of symptoms) among people with schizophrenia and their caregivers could also be an interesting target for future research.

Conclusion

Providing that the instruments are laid out clearly, instructions are understood, and the device is easy to use, eCOA is a feasible method of on-site data collection in people with schizophrenia and their caregivers. This appears to be the case regardless of clinical or demographic characteristics of respondents (eg, schizophrenia severity, age, or computer familiarity). In addition, research support staff members can easily assist people with schizophrenia and their caregivers in completing assessments on an eCOA device.

Acknowledgment

The Schizophrenia Caregiver Questionnaire (SCQ) and the Caregiver Global Impression Scales (CaGI), including but not limited to all and any translations and other derivatives (eg, electronic versions) are protected by international copyright: SCQ ©, CaGI © F. Hoffmann-La Roche Ltd & Professor Zarit 2012. For information on, or permission to use, the SCQ or CaGI, please contact the Mapi Research Trust. Email: [email protected]. Website: http://www.proqolid.org.

Disclosure

This project was funded by F. Hoffmann-La Roche Ltd, who commissioned Adelphi Values, a health outcomes agency by whom CT and AG are employed. DR and JKL are both employees of F. Hoffmann-La Roche Ltd. The authors report no other conflicts of interest in this work.

---

References

1.	Gater A, Rofail D, Tolley C, et al (2014). Sometimes it’s difficult to have a normal life: results from a qualitative study exploring caregiver burden in schizophrenia. Schizophr Res Treatment. Epub April 3, 2014.
2.	Coons SJ, Gwaltney CJ, Hays RD, et al; ISPOR ePRO Task Force. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health. 2009;12:419–429.
3.	Grignon S, Grégoire CA, Durand M, Mury M, Elie D, Chianetta JM. Age-dependent discrepancies between computerized and paper cognitive testing in patients with schizophrenia. Soc Psychiatry Psychiatr Epidemiol. 2009;44:73–77.
4.	Gwaltney CJ, Shields A, Shiffman S. Equivalence of electronic and paper - and - pencil administration of patient - reported outcome measures: a meta-analytic review. Value in Health. 2008;11(2):322–333.
5.	Ben-Zeev D, Davis KE, Kaiser S, Krzsos I, Drake RE. Mobile technologies among people with serious mental illness: opportunities for future services. Adm Policy Ment Health. 2013;40(4):340–343.
6.	Saraceno B, Levav I, Kohn R. The public mental health significance of research on socio-economic factors in schizophrenia and major depression. World Psychiatry. 2005;4:181–185.
7.	Green MF. Cognitive impairment and functional outcome in schizophrenia and bipolar disorder. J Clin Psychiatry. 2006;67:e12.
8.	US Census Bureau. Computer and Internet Use in the United States; 2013. Available from: http://www.census.gov/content/dam/Census/library/publications/2014/acs/acs-28.pdf. Accessed February 10, 2015.
9.	Bozionelos N. Socio-economic background and computer use: the role of computer anxiety and computer experience in their relationship. Int J Hum Comput Stud. 2004;61:725–746.
10.	Granholm E, Loh C, Swendsen J. Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophr Bull. 2008;34:507–514.
11.	Johnson EI, Grondin O, Barrault M, et al. Computerized ambulatory monitoring in psychiatry: a multi-site collaborative study of acceptability, compliance, and reactivity. Int J Methods Psychiatr Res. 2009;18: 48–57.
12.	Swendsen J, Ben-Zeev D, Granholm E. Real-time electronic ambulatory monitoring of substance use and symptom expression in schizophrenia. Am J Psychiatry. 2011;168:202–209.
13.	Caregiver Law and Legal Definition [webpage on the Internet]. NRS. Nevada Revised Statutes 453A.080; 2012. Available from: http://definitions.uslegal.com/c/caregiver/. Accessed February 10, 2015.
14.	Wilkinson G, Hesdon B, Wild D, et al. Self-report quality of life measure for people with schizophrenia: the SQLS. Br J Psychiatry. 2000;177:42–46.
15.	Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med. 2001;33(5):337–343.
16.	Ware JE Jr. SF-36 health survey update. Spine (Phila Pa 1976). 2000;25(24):3130–3139
17.	Gater A, Rofail D, Tolley C et al. Assessing the Impact of Caring for a Person with Schizophrenia. Development of the Schizophrenia Caregiver Questionnaire. The Patient- Patient-Centered Outcomes Research (2015) 1–14.
18.	US Food and Drug Administration. Guidance for Industry. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiological Health (CDRH); 2009.
19.	Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18:59–82.
20.	Francis JJ, Johnston M, Robertson C, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25:1229–1245.
21.	Kimhy D, Delespaul P, Corcoran C, Ahn H, Yale S, Malaspina D. Computerized experience sampling method (ESMc):assessing feasibility and validity among individuals with schizophrenia. J Psychiatr Res. 2006;40:221–230.
22.	Kimhy D, Myin-Germeys I, Palmier-Claus J, Swendsen J. Mobile assessment guide for research in schizophrenia and severe mental disorders. Schizophr Bull. 2012;38:386–395.
23.	Arbuckle R, Tolley C, Burbridge C. How burdensome is completion of electronic patient-reported outcomes (ePRO)? Item completion times and qualitative evidence from studies in four different health conditions. Presented at: Value in Health Conference: 17th Annual International Meeting of the International Society for Pharmacoeconomics and Outcomes Research, ISPOR 16th Annual European Congress; November 2–6, 2013; Washington, DC. 16:A604.
24.	Pickard AS, Neary MP, Cella D. Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health Qual Life Outcomes. 2007;5:70.
25.	Stull DE, Leidy NK, Parasuraman B, Chassany O. Optimal recall periods for patient-reported outcomes: challenges and potential solutions. Curr Med Res Opin. 2009;25:929–942.
26.	Chinman M, Young AS, Schell T, Hassell J, Mintz J. Computer-assisted self-assessment in persons with severe mental illness. J Clin Psychiatry. 2004;65:1343–1351.