A survey of eMedia-delivered interventions for schizophrenia used in randomized controlled trials

Farooq Naeem; Tariq Munshi; Shuo Xiang; Megan Yang; Farhad Shokraneh; Yumeen Syed; Muhammad Ayub; Clive E Adams; Saeed Farooq

doi:10.2147/NDT.S115897

Back to Journals » Neuropsychiatric Disease and Treatment » Volume 13

Original Research

A survey of eMedia-delivered interventions for schizophrenia used in randomized controlled trials

Authors Naeem F, Munshi T, Xiang S, Yang M, Shokraneh F , Syed Y, Ayub M, Adams CE, Farooq S

Received 27 June 2016

Accepted for publication 12 October 2016

Published 27 January 2017 Volume 2017:13 Pages 233—243

DOI https://doi.org/10.2147/NDT.S115897

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Roger Pinder

Download Article [PDF]

Farooq Naeem,¹ Tariq Munshi,¹ Shuo Xiang,¹ Megan Yang,¹ Farhad Shokraneh,² Yumeen Syed,³ Muhammad Ayub,¹ Clive E Adams,⁴ Saeed Farooq⁵

¹Department of Psychiatry, Queens University, Kingston, ON, Canada; ²Institute of Mental Health, Nottingham, UK; ³Trenlogic, Toronto, ON, Canada; ⁴Nottingham University, Nottingham, ⁵Keele University, Stafford, UK

Background: Randomized trials evaluating electronic Media (eMedia) delivery of interventions are increasingly frequent in mental health. Although a number of reviews have reported efficacy of these interventions, none has reviewed the type of eMedia interventions and quality of their description. We therefore decided to conduct a survey of eMedia-delivered interventions for schizophrenia.
Methods: We surveyed all relevant trials reliably identified in the Cochrane Schizophrenia Group’s comprehensive register of trials by authors working independently. Data were extracted regarding the size of the trial, interventions, outcomes and how well the intervention was described.
Results: eMedia delivery of interventions is increasingly frequent in trials relevant to the care of people with schizophrenia. The trials varied considerably in sample sizes (mean =123, median =87, range =20–507), and interventions were diverse, rarely evaluating the same approaches and were poorly reported. This makes replication impossible. Outcomes in these studies are limited, have not been noted to be chosen by end users and seem unlikely to be easy to apply in routine care. No study reported on potential adverse effects or cost, end users satisfaction or ease of use. None of the papers mentioned the use of CONSORT eHealth guidelines.
Conclusion: There is a need to improve reporting and testing of psychosocial interventions delivered by eMedia. New trials should comply with CONSORT eHealth guidance on design, conduct and reporting, and existing CONSORT should be updated regularly, as the field is constantly evolving.

Keywords: randomized trials, survey, eMedia, information technology, schizophrenia, methodology

Background

The use of technology to advance psychological treatment is a developing professional reality.¹ The digital revolution has the potential to transform mental health care by connecting patients, services and health data in new ways.² There has been extensive research on the use of electronic Media (eMedia) technologies for a variety of emotional and mental health problems. A number of literature reviews of interventions delivered through eMedia have been published. These include reviews of interventions for anxiety disorders in adults,³ behavior problems in children,⁴ treatment of alcohol-related problems⁵ and patient education and support for people with schizophrenia.⁶

Numerous mobile phone-based telemedicine solutions for remote patient monitoring and disease management in schizophrenia have been tested.⁷ The eMedia-delivered interventions for this population group range from simple telephone reminders to attend follow-up appointments⁸ to complex medication monitoring systems to improve compliance.⁹ The virtual reality (VR) programs are being used to teach people various skills, most notably vocational and the social competencies,^10,11 but novel approaches have been developed to help people with chronic psychotic symptoms.¹²

eMedia

eMedia can be defined as the media that use electronics or electromechanical energy for end users (audience) to access content (this is in contrast to static media, mainly print). The eMedia sources most familiar to the general public are video recordings, audio recordings, multimedia presentations, slide presentations, CD-ROM and online content. Most new eMedia are now in a digital Media form (dMedia). Numerous terms are used to describe interventions delivered through dMedia, including mobile health and mobile psychosocial interventions (mPSI) (Table 1).

Table 1 Overall classification of media to deliver interventions for mental health

Reviews in this area address the effects of specific forms of eMedia,^3,13,14 but we know of no definitive overview of how eMedia has been used and described in a defined group of randomized controlled trials (RCTs). In order to replicate the findings, the interventions should be clearly described. This should involve details of their development and initial testing.¹⁵ The CONSORT guidelines recommend that in reporting of an RCT, the authors should describe each intervention thoroughly, including control interventions. The description should allow a clinician to know exactly how to administer the intervention that was evaluated in the trial. If the control group is to receive treatment-as-usual (TAU), it is imperative to describe thoroughly what that constitutes. If the control group or intervention group is to receive a combination of interventions, these should be thoroughly described.^16,17

Recently, attempts have been made to modify the CONSORT guidelines.¹⁸ However, these are not widely used, even though it has been claimed that implementing these might improve the reporting quality of trials in this area.¹⁹ The guidelines need to be continuously updated as our understanding of the issues surrounding eMedia interventions is constantly improving.^15,20

By taking an overview of the interventions in a specific area it may be possible to highlight areas that need further development in this new and exciting development in mental health care.

Aims

The aim of this survey was to provide a broad overview of the type of eMedia interventions for all randomized trials relevant to people with schizophrenia. We also wanted to examine whether these interventions are described in details using a criterion (for example, existing CONSORT eHealth checklist). We did not aim to review the effects of these interventions.

Methods

Search methods

Searched source

On January 6, 2015, we searched the Cochrane Schizophrenia Group’s Study-Based Register of Trials and updated this search on October 5, 2015. The Cochrane Schizophrenia Group’s Register of Trials is kept on Microsoft access-based computer program called MeerKat (version 1.6) and is compiled by systematic searches of major national and international resources (including monthly updates of AMED, BIOSIS, CINAHL, EMBASE, MEDLINE, PsycINFO, PubMed and registries of clinical trials) and their monthly updates, hand-searches, gray literature and conference proceedings (for detailed description of the register see Group’s Module). There is no language, date, document type or publication status limitations for the inclusion of records into this register.

Search strategy

We used the following search strategy, which has been developed based on literature review and consulting with the authors of the review:

(*Computer* OR *Internet* OR *Electronic* OR *Media* OR *Phone*) in Intervention Field of STUDY (418 results).

In such a study-based register, searching the major concept retrieves all the synonym keywords and relevant studies because all the studies have already been organized based on their interventions and linked to the relevant topics (Table S1 describes detailed keywords).

Inclusion criteria

We included published randomized or quasi-RCTs using any eMedia relevant to people with schizophrenia or schizophrenia-like illnesses with at least one outcome relevant to that same group of people. We also included studies that used social media or the Internet for patient or caregiver education. There was no requirement for papers to report any particular outcome measure or length of follow-up.

By eMedia, we meant media that use electronics or electromechanical energy for end users (audience) to access content (for example, interventions delivered through [but not limited to] mobile phones, tablets, computers, TV or radio). Initially, we wanted to classify interventions according to the type of media and type of e-platform being used, based on information technology (IT) system. IT experts advised that the interventions could be classified based on the extension and file type. However, it might be too complicated for health professionals (Table 2). On detailed examination of studies, however, we found that only limited information was available in the majority of studies on these aspects of intervention, and it was impossible to classify on this basis. In any event, this was also not clinically relevant and we, therefore, decided to classify studies according to the type of intervention.

Table 2 Types of digital media

Initial search returned 296 abstracts. Of these, 96 were considered irrelevant. All the abstracts were carefully inspected. Full paper was obtained for clarification if warranted. Given the lack of consensus criteria for eMedia, we adopted a broad definition whereby a paper was included if the intervention was deemed to be delivered by an electronic medium by the trialist. We also included papers in which the trialists did not explicitly mention the intervention, but there was evidence that some electronic medium was employed. Multiple reports of single trials were grouped to avoid double counting. Two authors (FN and SX) held numerous discussions to clarify selection, issues around media categories, suitability and classification of studies into groups. The selected trials were then grouped by type of intervention, and the data on the description of the intervention and size of the trial were independently extracted (FN, SX and TM). SF was consulted when there was a disagreement and to clarify issues. We worked with a team of IT specialists to clarify issues around the classification of eMedia types. One author (SX) has qualifications in IT and medicine. Data were entered into an electronic database. In addition, we also inspected published meta-analyses and relevant Cochrane reviews. After excluding duplicates, and studies not concerning a therapeutic intervention or assessment, we finally selected 61 studies.

Results

The 61 studies involved 5,352 people with schizophrenia (mean =123, median =87, range =20–507). Studies of this type are of increasing in frequency (Figure 1).

Figure 1 Number of RCTs over the years (2001–2014).
Abbreviation: RCTs, randomized control trials.

These trials used a variety of eMedia (eg, mobile/static telephone – calls/messaging/applications [apps], computer software, web pages) and reported outcomes such as adherence, assessment of symptoms, patient education, symptom improvements and compliance (Table 3).

Table 3 Summary of randomised trials of eMedia interventions for schizophrenia
Note: *Included in a Cochrane review.
Abbreviation: TAU, Treatment As Usual.

We classified the interventions according to the target areas into the following types:

Education and information
Assessment, screening and monitoring
Improved communications
Relapse management/engagement
Specialized interventions

Eleven trials reported testing of eMedia-delivered education/information or training (mean =124, median =91, range =25–311): four on assessment, screening or symptom monitoring (mean =100, median =90, range =24–225), another four on communication improvement (mean =221, median =165, range =50–507) and six on relapse management and engagement (mean =135, median =124, range =47–254). However, the largest number of trials (37) focused on testing of provision of specialized interventions. These included cognitive remediation (CR) (34 RCTs, N=1,833), problem solving (one RCT, N=20) and a novel cognitive behaviour therapy [CBT]-based Avatar therapy (one RCT, N=26).

TAU was the most common control group. Only a small number of studies compared the intervention with an active control (eg, an educational training video compared with a non-educational control video) or with a real-life intervention (eg, VR job skills training compared with the therapist-delivered group). Only two studies compared the same intervention delivered through a different eMedia (eg, SMS-delivered problem solving compared with telephone-delivered problem solving). None of the trials surveyed in this study mentioned the use of CONSORT-eHEALTH checklist. These papers did not describe the interventions in details using an existing criterion. Although a significant minority of studies reported feasibility of the interventions (n=12), none reported end user experience. None of the trials reported participants’ level of digital literacy.²⁰

Discussion

Nearly half (n=35) of the studies reported the use of computer programs for CR. Trials of eMedia-delivered interventions are increasing in frequency. Studies that may have seemed unusual a decade ago are now much more common.

Most studies are small (please note the lower end of the range of numbers in “Results” section). We recognize that many were undertaken by researchers working on very low budgets, and power calculations may have been uninformative when there is little to base them on. Now this does not apply. There are studies albeit small. More informative power calculations are possible, and we suggest that these should be for outcomes that end users designate as important. Even if studies of the exact eMedia do not exist, those proposing to undertake such trials should be able to extrapolate from other good evidence – even if the original intervention was not delivered by eMedia.

To our knowledge, this is the first literature survey reporting an overview of eMedia-delivered interventions for schizophrenia. Previous literature reviews have focused mainly on the outcome of interventions.^3–6,13,21 The eMedia interventions, like any other health care intervention, should be clearly described so that results can be replicated.¹⁵

Types of eMedia

There was a wide variation in the eMedia used. It varied from television (TV), radio, telephone and desktop computer to game console, smartphones and handheld device. It is not possible to classify eMedia due to the variety of file formats. On the basis of this survey, currently there are two types of media used:

Interactive media (eg, mobile phone apps or websites)
Didactic media (TV, video, etc.)

The survey also highlighted the following types of interventions that are delivered by eMedia;

Type 1: Intervention delivered by a human therapist through a mobile app or a computer program (eg, telephone-delivered problem solving by a therapist, Avatar therapy).

Type 2: Intervention based on a manualized, well-established therapy delivered through eMedia (eg, MoodGym).

Type 3: Intervention that did not exist before and is not underpinned by an existing theoretical framework.

Standardizing, regulating, guiding regarding the intervention

There is an interest in standardizing interventions in this area and have been attempts to establish criteria to self-certify mobile app development.²² This followed concerns about the lack of standardization and risk assessment of medical apps.^23,24

The health regulatory agencies have not paid sufficient attention to eMedia-delivered interventions, and therefore limited information and guidance are available for clinicians or end users. The US Food and Drug Administration (FDA) released guidance recently, focusing primarily on apps that transform the mobile platform into a regulated medical device.²⁵ The remaining eMedia will be subject to “enforcement discretion”, that is, no regulation.²⁶ The Therapeutic Goods Administration for health care practitioners in Australia provides limited guidance.²⁷ Similarly, the British Medicines and Healthcare Products Regulatory Agency does not offer concrete advice for clinicians or end users. However, the Business Standards Institution has developed “PAS 277 Health and wellness apps – Quality criteria across the life cycle – Code of practice”²⁸ in conjunction with Innovate UK (www.innovateuk.gov.uk). The purpose of these guidelines is to develop a set of principles that health app developers should follow to ensure that their products and services can be trusted by health care professionals and the public. This is an important starting point. However, with the slow recognition, acceptance and standardization of health apps and the fast pace of development, it seems unlikely that standards and regulation will keep up with technological advancement.²⁹ No trial reported ease (or difficulty) of access and use of the eMedia under test.

None of the studies in this survey followed a set of guidelines for quality control.²⁷ Having read these studies in detail, we remain unclear about the fidelity of most interventions. A small number of studies describe the evidence relating to the manualized intervention, which is, in their trial, delivered through an eMedia (for example, CR trials). A few studies gave details of the eMedia developers, and an even smaller number gave details of the intervention in the public arena (for example, https://www.itareps.com). While some of these interventions might be ready to be used in clinical practice (eg, ClinTouch), others may well be impractical (eg, Avatar therapy) due to cost, training or ease/difficulty of use.

These relatively new interventions present trialists with a dilemma of what to use as a comparison. TAU can be problematic,³⁰ and most studies using a TAU comparison failed to describe adequately what exactly was “usual”. A small number of trials employed innovative control groups such as an active control using a similar eMedia, a real-life intervention or a different eMedia used to deliver the same intervention.

Reporting and outcomes

Use of technical (IT) jargon may not make sense to many health professionals. This could easily be overcome by providing details of the intervention in an online supplement or website. Some guidance is available in reporting mPSI.¹⁵

Outcomes were not clearly of importance to end users. Scale-derived measures (for example, scores on Positive and Negative Syndrome Scale) are important mostly to researchers and are not usual in routine care. While researchers are an important constituency for the trial to serve, they are not the only one. End users might be more interested in an outcome like user experience. It is important that eMedia trials do not lose the opportunity to undertake trials that are likely to make any effective procedure applicable to wide use.²⁹

None of the studies reported on direct or indirect adverse effects or on the cost-effectiveness of these interventions (or just the cost of the intervention). Effective encouragement of taking medications may lead to adverse events. There is evidence that psychosocial interventions also cause adverse outcomes.³¹ Delivery through eMedia complicates and possibly increases risks. Furthermore, there may be risks particular to eMedia such as reduction in social activities or even addiction to eMedia.²³

There seems to be an underlying assumption in developing eMedia-delivered interventions that they are cost-effective compared with human media-delivered treatments. No study reported cost per unit of intervention or cost-effectiveness. The assumption of eMedia being more cost-effective remains an assumption.

Criteria

There is an urgent need to set criteria for standardization of development, evaluation and reporting of eMedia interventions. It is imperative that uniform processes are followed and that criteria for trial reporting are developed in consultation with experts in care, evaluation and IT. Recognizing that such suggestions must evolve with technology, we suggest that the CONSORT statement needs to be constantly updated.

Conclusion

This survey reports numerous areas of weakness in current description of eMedia-delivered interventions for schizophrenia. Most notable issues are descriptions of development and initial testing and reporting of interventions, their theoretical underpinning and their side effects. There is also a need to improve reporting end user experience and adapting interventions to their digital or e-literacy skills. None of the trials reported use of CONSORT eHealth. Updating these guidelines, especially for psychosocial interventions, might be the way forward.

Acknowledgment

We are grateful to the Cochrane Schizophrenia Group for their support.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Barak A, Hen L, Boniel-Nissim M, Shapira N. A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions. National Library of Medicine. PubMed Health. 2008 [cited 2015 Sep 26]. Available from: http://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0026877/. Accessed May 20, 2016.
2.		Hollis C, Morriss R, Martin J, et al. Technological innovations in mental healthcare: harnessing the digital revolution. Br J Psychiatry. 2015;206(4):263–265.
3.		Mayo-Wilson E, Montgomery P. Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults. Cochrane Database Syst Rev. 2013;(9):CD005330.
4.		Montgomery P. Media-based behavioural treatments for behavioural problems in children. Cochrane Database Syst Rev. 2006;(1):CD002206.
5.		Stallman H, Kavanagh D, White A. Internet interventions for treatment of alcohol-related problems. The Cochrane Library. Stallman. Wiley Online Library. [cited 2015 Sep 26]. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010228/abstract. Accessed May 20, 2016.
6.		Välimäki M, Hätönen H, Lahti M, Kuosmanen L, Adams CE. Information and communication technology in patient education and support for people with schizophrenia. Cochrane Database Syst Rev. 2012;10:CD007198.
7.		Španiel F, Vohlídka P, Kožený J, et al. The Information Technology Aided Relapse Prevention Programme in Schizophrenia: an extension of a mirror-design follow-up. Int J Clin Pract. 2008;62(12):1943–1946.
8.		Burgoyne RW, Acosta FX, Yamamoto J. Telephone prompting to increase attendance at a Psychiatric Outpatient Clinic. Am J Psychiatry. 1983;140(3):345–347.
9.		Frangou S, Sachpazidis I, Stassinakis A, Sakas G. Telemonitoring of medication adherence in patients with schizophrenia. Telemed J E Health. 2005;11(6):675–683.
10.		Park KM, Ku J, Choi SH, et al. A virtual reality application in role-plays of social skills training for schizophrenia: a randomized, controlled trial. Psychiatry Res. 2011;189(2):166–172.
11.		Smith MJ, Fleming MF, Wright MA, et al. Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophr Res. 2015;166(1–3):86–91.
12.		Leff J, Williams G, Huckvale MA, Arbuthnot M, Leff AP. Computer-assisted therapy for medication-resistant auditory hallucinations: proof-of-concept study. Br J Psychiatry. 2013;202(6):428–433.
13.		Clement S, Lassman F, Barley E, et al. Mass media interventions for reducing mental health-related stigma. Cochrane Database Syst Rev. 2013;(7):CD009453.
14.		Olthuis JV, Watt MC, Stewart SH. Therapist-delivered distance cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev. John Wiley & Sons, Ltd; 2011 [cited 2015 Sep 27]. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009028/abstract. Accessed May 20, 2016.
15.		Naeem F, Syed Y, Xiang S, et al. Development, testing and reporting of mobile apps for psycho-social interventions: lessons from the pharmaceuticals. J Med Diagn Methods. 2015 [cited 2016 Jan 18];4(1000191). Available from: http://www.omicsgroup.org/journals/development-testing-and-reporting-of-mobile-apps-for-psychosocialinterventions-lessons-from-the-pharmaceuticals-2168-9784-1000191.php?aid=65355. Accessed May 20, 2016.
16.		Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P; CONSORT Group. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148(4):295–309.
17.		Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8(1):18.
18.		Eysenbach G; CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res. 2011;13(4):e126.
19.		Eysenbach G. CONSORT-EHEALTH: implementation of a checklist for authors and editors to improve reporting of web-based and mobile randomized controlled trials. Stud Health Technol Inform. 2013;192:657–661.
20.		Naeem F, Gire N, Xiang S, et al. Reporting and understanding the safety and adverse effect profile of mobile apps for psychosocial interventions: an update. World J Psychiatry. 2016;6(2):187–191.
21.		Klecun E, Lichtner V, Cornford T. e-Literacy in health care. Stud Health Technol Inform. 2014;205:843–847.
22.		White A, Kavanagh D, Stallman H, et al. Online alcohol interventions: a systematic review. J Med Internet Res. 2010;12(5):e62.
23.		Lewis T. A systematic self-certification model for mobile medical apps. J Med Internet Res. 2013;15(4):e89.
24.		Lewis TL, Wyatt JC. mHealth and mobile medical apps: a framework to assess risk and promote safer use. J Med Internet Res. 2014;16(9):e210.
25.		FDA. Mobile medical applications [Internet]. 2014 [cited 2015 Oct 15]. Available from: http://www.fda.gov/MedicalDevices/DigitalHealth/MobileMedicalApplications/default.htm. Accessed May 20, 2016.
26.		McCarthy M. FDA will not regulate most mobile medical apps. BMJ. 2013;347:f5841.
27.		TGA. Regulation of medical software and mobile medical “apps” [Internet]. Therapeutic Goods Administration (TGA). 2013 [cited 2015 Oct 15]. Available from: https://www.tga.gov.au/regulation-medical-software-and-mobile-medical-apps. Accessed May 20, 2016.
28.		BSI. PAS 277:2015 Health and wellness apps. Quality criteria across the life cycle. Code of practice [Internet]. 2015 [cited 2015 Oct 15]. Available from: http://shop.bsigroup.com/forms/PASs/PAS-2772015/. Accessed May 20, 2016.
29.		Vincent CJ, Niezen G, O’Kane AA, Stawarz K. Can standards and regulations keep up with health technology? JMIR MHealth UHealth. 2015;3(2):e64.
30.		Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464–475.
31.		Berk M, Parker G. The elephant on the couch: side effects of psychotherapy. Aust N Z J Psychiatry. 2009;43(9):787–794.
32.		Rotondi AJ, Haas GL, Anderson CM, et al. A Clinical Trial to Test the Feasibility of a Telehealth Psychoeducational Intervention for Persons With Schizophrenia and Their Families: Intervention and 3-Month Findings. Rehabil Psychol. 2005;50(4):325–336.
33.		Rotondi AJ, Anderson CM, Haas GL, et al. Web-based Home Delivery of Multi-Family Psychoeducational Therapy to Persons with Schizophrenia and their Family Members: 1-year Outcomes and Cognitive Design. In: Schizophrenia Bulletin. Oxford Univ Press Great Clarendon St, Oxford Ox2 6dp, England; 2011. p. 319– 319.
34.		Wirshing DA, Sergi MJ, Mintz J. A Videotape Intervention to Enhance the Informed Consent Process for Medical and Psychiatric Treatment Research. Am J Psychiatry. 2005;162(1):186–188.
35.		Jones RB, Atkinson JM, Coia DA, et al. Randomised trial of personalised computer based information for patients with schizophrenia. BMJ. 2001;322(7290):835–840.
36.		Pitkänen A, Välimäki M, Kuosmanen L, et al. Patient education methods to support quality of life and functional ability among patients with schizophrenia: a randomised clinical trial. Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 2012;21(2):247–256.
37.		Kaplan K, Solomon P, Salzer MS, Brusilovskiy E. Assessing an Internet-based parenting intervention for mothers with a serious mental illness: A randomized controlled trial. Psychiatr Rehabil J. 2014;37(3):222–231.
38.		Park K-M, Ku J, Choi S-H, et al. A virtual reality application in role-plays of social skills training for schizophrenia: a randomized, controlled trial. Psychiatry Res. 2011;189(2):166–172.
39.		Smith MJ, Fleming MF, Wright MA, et al. Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophr Res. 2015;166(1–3):86–91.
40.		Tsang MMY, Man DWK. A virtual reality-based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophr Res. 2013;144(1–3):51–62.
41.		Torres A, Mendez LP, Merino H, Moran EA. Improving social functioning in schizophrenia by playing the train game. Psychiatr Serv Wash DC. 2002;53(7):799–801.
42.		Kaplan K, Salzer MS, Solomon P, Brusilovskiy E, Cousounis P. Internet peer support for individuals with psychiatric disabilities: A randomized controlled trial. Soc Sci Med. 2011;72(1):54–62.
43.		Chinman M, Young AS, Schell T, Hassell J, Mintz J. Computer-assisted self-assessment in persons with severe mental illness. J Clin Psychiatry. 2004;65(10):1343–1351.
44.		Anzai N. Rehab Rounds: Training Persons With Schizophrenia in Illness Self-Management: A Randomized Controlled Trial in Japan. Psychiatr Serv. 2002;53(5):545–547.
45.		Kasckow J, Zickmund S, Gurklis J, et al. Using telehealth to augment an intensive case monitoring program in veterans with schizophrenia and suicidal ideation: A pilot trial. Psychiatry Res. 2016;239:111–116.
46.		Palmier-Claus JE, Ainsworth J, Machin M, et al. The feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry. 2012;12:172.
47.		van der Krieke L, Emerencia AC, Boonstra N, Wunderink L, de Jonge P, Sytema S. A Web-Based Tool to Support Shared Decision Making for People With a Psychotic Disorder: Randomized Controlled Trial and Process Evaluation. J Med Internet Res. 2013;15(10):e216.
48.		Steinwachs DM, Roter DL, Skinner EA, et al. A web-based program to empower patients who have schizophrenia to discuss quality of care with mental health providers. Psychiatr Serv Wash DC. 2011;62(11):1296–1302.
49.		Priebe S, McCabe R, Bullenkamp J, et al. Structured patient-clinician communication and 1-year outcome in community mental healthcare: cluster randomised controlled trial. Br J Psychiatry J Ment Sci. 2007;191:420–426.
50.		Woltmann EM, Wilkniss SM, Teachout A, McHugo GJ, Drake RE. Trial of an Electronic Decision Support System to Facilitate Shared Decision Making in Community Mental Health. Psychiatr Serv. 2011;62(1):54–60.
51.		Zanjani F, Miller B, Turiano N, Ross J, Oslin D. Effectiveness of Telephone-Based Referral Care Management, a Brief Intervention to Improve Psychiatric Treatment Engagement. Psychiatr Serv. 2008;59(7):776–781.
52.		Komatsu H, Sekine Y, Okamura N, et al. Effectiveness of Information Technology Aided Relapse Prevention Programme in Schizophrenia excluding the effect of user adherence: a randomized controlled trial. Schizophr Res. 2013;150(1):240–244.
53.		Stip EM, Vincent PDP, Guevremont CMS, et al. A randomized controlled trial with a Canadian electronic pill dispenser used to measure and improve medication adherence in patients with schizophrenia. Pharm Med Outcomes Res. 2013;4:100.
54.		Frangou S, Sachpazidis I, Stassinakis A, Sakas G. Telemonitoring of medication adherence in patients with schizophrenia. Telemed J E-Health Off J Am Telemed Assoc. 2005;11(6):675–683.
55.		Velligan D, Mintz J, Maples N, et al. A randomized trial comparing in person and electronic interventions for improving adherence to oral medications in schizophrenia. Schizophr Bull. 2013;39(5):999–1007.
56.		Kauppi K, Kannisto KA, Hätönen H, et al. Mobile phone text message reminders: Measuring preferences of people with antipsychotic medication. Schizophr Res. 2015;168(1–2):514–522.
57.		Bryson G, Bell MD. Initial and final work performance in schizophrenia: cognitive and symptom predictors. J Nerv Ment Dis. 2003;191(2):87–92.
58.		Cavallaro R, Anselmetti S, Poletti S, et al. Computer-aided neurocognitive remediation as an enhancing strategy for schizophrenia rehabilitation. Psychiatry Res. 2009;169(3):191–196.
59.		d’Amato T, Bation R, Cochet A, et al. A randomized, controlled trial of computer-assisted cognitive remediation for schizophrenia. Schizophr Res. 2011;125(2–3):284–290.
60.		Dang J, Zhang J, Guo Z, et al. A pilot study of iPad-assisted cognitive training for schizophrenia. Arch Psychiatr Nurs. 2014;28(3):197–199.
61.		Dickinson D, Tenhula W, Morris S, et al. A Randomized, Controlled Trial of Computer-Assisted Cognitive Remediation for Schizophrenia. Am J Psychiatry. 2010;167(2):170–180.
62.		Drake RJ, Day CJ, Picucci R, et al. A naturalistic, randomized, controlled trial combining cognitive remediation with cognitive-behavioural therapy after first-episode non-affective psychosis. Psychol Med. 2014;44(9):1889–1899.
63.		Garrido G, Barrios M, Penadés R, et al. Computer-assisted cognitive remediation therapy: Cognition, self-esteem and quality of life in schizophrenia. Schizophr Res. 2013;150(2–3):563–569.
64.		Garrido G, Barrios M, Penadés R. Cognition, Self-esteem and Quality of Life in Schizophrenia; A 12 Month Follow-up Cognitive Remediation Clinical Trial. Schizophr Res. 2014;153:S257–S258.
65.		Harris A, Rogers P, Moore G, Toscano D, Redoblado-Hodge A. Cognitive remediation in first-episode and chronic schizophrenia. Acta Neuropsychiatr. 2006;18(6):253.
66.		Fisher M, Holland C, Merzenich MM, Vinogradov S. Using Neuroplasticity-Based Auditory Training to Improve Verbal Memory in Schizophrenia. Am J Psychiatry. 2009;166(7):805–811.
67.		Holzer L, Urben S, Passini CM, et al. A randomized controlled trial of the effectiveness of computer-assisted cognitive remediation (CACR) in adolescents with psychosis or at high risk of psychosis. Behav Cogn Psychother. 2014;42(4):421–434.
68.		Holzer L, Urben S, Pihet S, Jaugey L. A randomized controlled trial of the effectiveness of a computer-assisted cognitive remediation (CACR) program in adolescents with psychosis or at high risk of psychosis: Short-term and long-term outcomes. Neuropsychiatr Enfance Adolesc. 2012;60(5):S71.
69.		Torrisi R, Holzer L, Pihet S, et al. Computer-assisted cognitive remediation program for adolescents at high risk of psychosis or with psychotic disorders: preliminary results. Int Clin Psychopharmacol. 2011;26:e85–e86.
70.		Hooker C, Bruce L, Fisher M, Verosky S, Miyakawa A, Vinogradov S. Neural activity during emotion recognition after combined cognitive plus social cognitive training in schizophrenia. Schizophr Res. 2012;139(1–3).
71.		Hooker CI, Bruce L, Fisher M, Verosky SC, Miyakawa A, Vinogradov S. Neural activity during emotion recognition after combined cognitive plus social cognitive training in schizophrenia. Schizophr Res. 2012;139(1):53–59.
72.		Horan WP, Kern RS, Tripp C, et al. Efficacy and specificity of Social Cognitive Skills Training for outpatients with psychotic disorders. J Psychiatr Res. 2011;45(8):1113–1122.
73.		Jaugey L. Short-and Long-Term Outcomes of a Randomized Controlled Trial of a Computer-Assisted Cognitive Remediation (CACR) Program in Adolescents with Psychosis or at High Risk of Psychosis. Biol Psychiatry. 2012;71(8):84s–84s.
74.		Lee WK. Effectiveness of computerized cognitive rehabilitation training on symptomatological, neuropsychological and work function in patients with schizophrenia. Asia-Pac Psychiatry. 2013;5(2):90–100.
75.		Keefe RSE, Vinogradov S, Medalia A, et al. Feasibility and Pilot Efficacy Results From the Multisite Cognitive Remediation in the Schizophrenia Trials Network (CRSTN) Randomized Controlled Trial. J Clin Psychiatry. 2012;73(7):1016–1022.
76.		Bucci P, Piegari G, Mucci A, et al. Neurocognitive individualized training versus social skills individualized training: A randomized trial in patients with schizophrenia. Schizophr Res. 2013;150(1):69–75.
77.		Piskulic D, Barbato M, Liu L, Addington J. Pilot study of cognitive remediation therapy on cognition in young people at clinical high risk of psychosis. Psychiatry Res. 2015;225(1–2):93–98.
78.		Piskulic D, Barbato M, Addington J. Effects of Cognitive Remediation on Cognition in Young People at Clinical High Risk of Psychosis. Schizophrneia Res. 153:S128.
79.		Poletti S, Anselmetti S, Bechi M, et al. Computer-aided neurocognitive remediation in schizophrenia: durability of rehabilitation outcomes in a follow-up study. Neuropsychol Rehabil. 2010;20(5):659–674.
80.		Rass O, Forsyth JK, Bolbecker AR, et al. Computer-assisted cognitive remediation for schizophrenia: A randomized single-blind pilot study. Schizophr Res. 2012;139(1–3):92–98.
81.		Stürz K, Hartmann S, Eder-Pelzer B, Günther V. [Computer assisted cognitive training advances mood and psychological wellbeing – a comparison to paper pencil training relating to neuropsychological parameters, mood and cognitions]. ResearchGate. 2011;25(2):85–92.
82.		Vesterager L, Christensen TØ, Olsen BB, et al. Cognitive training plus a comprehensive psychosocial programme (OPUS) versus the comprehensive psychosocial programme alone for patients with first-episode schizophrenia (the NEUROCOM trial): A study protocol for a centrally randomised, observer-blinded multi-centre clinical trial. Trials [Internet]. 2011 Dec [cited 2016 Aug 15];12(1). Available from: http://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-12-35
83.		Vita A, De Peri L, Barlati S, et al. Effectiveness of different modalities of cognitive remediation on symptomatological, neuropsychological, and functional outcome domains in schizophrenia: A prospective study in a real-world setting. Schizophr Res. 2011;133(1–3):223–231.
84.		Ahmed AO, Hunter KM, Goodrum NM, et al. A randomized study of cognitive remediation for forensic and mental health patients with schizophrenia. J Psychiatr Res. 2015;68:8–18.
85.		Bell M, Bryson G, Greig T, Corcoran C, Wexler BE. Neurocognitive enhancement therapy with work therapy: effects on neuropsychological test performance. Arch Gen Psychiatry. 2001;58(8):763–768.
86.		McGurk SR, Mueser KT, Feldman K, Wolfe R, Pascaris A. Cognitive Training for Supported Employment: 2–3 Year Outcomes of a Randomized Controlled Trial. Am J Psychiatry. 2007;164(3):437–441.
87.		McGurk SR, Mueser KT, Pascaris A. Cognitive training and supported employment for persons with severe mental illness: one-year results from a randomized controlled trial. Schizophr Bull. 2005;31(4):898–909.
88.		Grynszpan O, Perbal S, Pelissolo A, et al. Efficacy and specificity of computer-assisted cognitive remediation in schizophrenia: a meta-analytical study. Psychol Med. 2011;41(1):163–173.
89.		Lecardeur L, Stip E, Giguere M, Blouin G, Rodriguez J-P, Champagne-Lavau M. Effects of cognitive remediation therapies on psychotic symptoms and cognitive complaints in patients with schizophrenia and related disorders: a randomized study. Schizophr Res. 2009;111(1–3):153–158.
90.		Hermanutz M, Gestrich J. Computer-assisted attention training in schizophrenics. A comparative study. Eur Arch Psychiatry Clin Neurosci. 1991;240(4–5):282–287.
91.		Beebe L, Smith KD, Phillips C. A Comparison of Telephone and Texting Interventions for Persons with Schizophrenia Spectrum Disorders. Issues Ment Health Nurs. 2014;35(5):323–329.
92.		Leff J, Williams G, Huckvale MA, Arbuthnot M, Leff AP. Computer-assisted therapy for medication-resistant auditory hallucinations: proof-of-concept study. Br J Psychiatry. 2013;202(6):428–433.

Supplementary material

Table S1 Detailed keywords

Creative Commons License © 2017 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.

Download Article [PDF]