The RECORD reporting guidelines: meeting the methodological and ethical demands of transparency in research using routinely-collected health data

Introduction

Information stored in repositories of routinely-collected health data (RCD) – such as health administrative datasets¹ – is increasingly regarded as a potential data source for clinical epidemiological research. The reasons are threefold: 1) data collection platforms are increasingly available for a wide range of data types; 2) with a greater number of sources, the volume of data is concomitantly growing, leading to greater breadth and depth of available data; and 3) primary data collection is increasingly costly, making secondary data analyses potentially cost-effective.

A number of funding agencies, such as the Canadian Institutes of Health Research, have actively endorsed the use of RCD for research, specifically for enhancing patient-oriented research and improving health care effectiveness, safety, and delivery.² Given these potential benefits, RCD are now used for a wide range of studies, including observational studies, comparative effectiveness research, diagnostics, studies of adverse effects, and predictive analytics.^3,4

At the same time, limitations inherent in using data collected without specific a priori research questions are increasingly recognized.^5,6 Concerns have been raised about data errors,⁷ missing data,^8,9 uncontrolled confounding,^10,11 data that are out of date, and data dredging.¹² Moreover, the potential linkage between datasets creates “myriad opportunities for the introduction of errors and omissions.”³ The potential for bias is amplified when linkage methods are inaccurate or incomplete,⁴ introducing errors that could have substantial consequences.³ There is also a growing awareness of the suboptimal quality of reports presenting research based on RCD.^13–15 This has created a “perfect storm” of increased interest and use of RCD for research, together with inadequate reporting of the strengths and weaknesses of these data resources.

Multifaceted benefits of improved reporting

Improving the reporting of studies using RCD not only facilitates comprehension and evaluation among readers but also allows replication of studies.¹⁶ Adequate documentation thus is a core standard of reporting.¹⁷ It is also a central ethical principle of clinical research. For example, the Declaration of Helsinki states that researchers have a duty to make the results of their research available and to do so in accordance with accepted guidelines for ethical reporting.¹⁸ This accords with respect for fairness and reciprocity: researchers draw on and should contribute to the accumulating pool of scientific knowledge.¹⁸

With research using RCD, transparent and accurate reporting may have the additional benefit of reducing research waste. Currently, research oversight is lacking for many sources of RCD. While data custodians may require approval procedures, there is no associated review of study questions or research methods to ensure efficient data use and prevent unnecessary duplication of analyses.^19,20

Therefore, accurate and complete reporting is needed to evaluate the clinical validity and utility of findings^4,8 and reduce duplication of effort. Adequate reporting of research also benefits by providing an external indicator that researchers are honest and trustworthy. Reporting guidelines provide a standard – a set of de facto professional norms – against which research can be judged. Investigators who are compliant with reporting guidelines fulfill their “social licence”²¹ to conduct research, demonstrating to the research community and broader public that they are satisfying its ethical requirements.

RECORD: meeting the research and ethics mandate for studies using RCD

This line of reasoning of course raises a key question: what standards should researchers and publishers uphold concerning RCD? Previously, the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement²² served as the standard, given that most research using RCD is observational. However, it has been acknowledged that the use of RCD raises additional issues not covered by STROBE, including description of the databases and validation of diagnostic codes.^23,24 The REporting of studies Conducted using Observational Routinely-collected Data (RECORD) statement²⁵ was developed to address these limitations and help researchers using RCD to meet their ethical obligations.

RECORD consists of a checklist of 13 items that supplement or modify existing STROBE items concerning an article’s title, abstract, introduction, methods, results, and discussion sections, as well as other information required in research reports. The recommendations reflect three broad areas of concern: identification of studies using RCD; evaluation of important methodological components; and information regarding access to and limits imposed on the data. Identifying research as using RCD is also important given the present lack of Medical Subject Heading terms with which to search for these types of studies. Identifying studies is a prerequisite to critiquing and building upon them.

The methods used to develop the RECORD guidelines were published in May 2015²⁶ and the full guideline appeared in October 2015.²⁵ Since then, both reports have been embraced by the scientific community. The RECORD statement has been endorsed by more than 15 journals, including Clinical Epidemiology. This journal now recommends that authors submit the RECORD checklist together with any manuscript reporting on research using RCD.

We consider this progress as just a beginning. We actively encourage ongoing discussion of the RECORD document from interested parties. An open discussion forum has been created within the RECORD statement website (http://www.record-statement.org/forum/)²⁷ as a place for interested individuals and groups to provide comments. We anticipate that this will lead to thoughtful contributions and possible future revisions of the checklist. As such, RECORD represents a living document that can be adapted to reflect changes in the field.

We will also continue to monitor the impact of the RECORD document on the field of clinical epidemiology. Studies of existing reporting guidelines suggest that endorsement by a journal leads to improved adherence among studies published within the journal. Moreover, use of checklists has been linked to improvements in completeness of research reports^28–32 and the quality of published articles.³³ Demonstrating the effectiveness of reporting guidelines is an important step in providing evidence of the benefits of RCD-based research to funding agencies as well as to the broader public and helping researchers justify their work in an era of increasing financial constraints.

Acknowledgments

Doctor Sørensen is supported by the Program for Clinical Research Infrastructure (PROCRIN) established by the Lundbeck Foundation and the Novo Nordisk Foundation. Doctor Langan is supported by an NIHR Clinician Scientist Fellowship (grant number: NIHR/CS/010/014). The findings and conclusions in this editorial are those of the authors and do not necessarily represent the views of the UK Department of Health. This article presents independent research funded in part by the National Institute for Health Research (NIHR).

Doctor Benchimol serves as cochair of the RECORD Steering Committee. He is supported by a New Investigator Award from the Canadian Institutes of Health Research, Canadian Association of Gastroenterology, and Crohn’s and Colitis Canada.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Council of Canadian Academies. Accessing Health and Health-Related Data in Canada. The Expert Panel on Timely Access to Health and Social Data for Health Research and Health System innovation. Ottawa, Canada: Council of Canadian Academies; 2015.
2.		Canadian Institutes of Health Research. Canada’s Strategy for Patient-Oriented Research. Improving health outcomes through evidence-informed care. Ottawa, Canada: Canadian Institutes of Health Research; 2011.
3.		Hoffman S, Podgurski A. The use and misuse of biomedical data: is bigger really better? Am J Law Med. 2013;39(4):497–538.
4.		Cohen IG, Amarasingham R, Shah A, Xie B, Lo B. The legal and ethical concerns that arise from using complex predictive analytics in health care. Health Aff. (Millwood). 2014;33(7):1139–1147.
5.		Dean BB, Lam J, Natoli JL, Butler Q, Aguilar D, Nordyke RJ. Review: use of electronic medical records for health outcomes research a literature review. Med Care Res Rev. 2009;66(6):611–638.
6.		Harpe SE. Using secondary data sources for pharmacoepidemiology and outcomes research. Pharmacotherapy. 2009;29(2):138–153.
7.		Guttmann A, Nakhla M, Henderson M, et al. Validation of a health administrative data algorithm for assessing the epidemiology of diabetes in Canadian children. Pediatr Diabetes. 2010;11(2):122–128.
8.		Kreuter F, Peng RD. Extracting information from big data: Issues of measurement, inference and linkage. In: Lane J, Stodden V, Bender S, Nissenbaum H, editors. Privacy, Big Data, and the Public Good: Frameworks for Engagement. Cambridge: Cambridge University Press; 2014:257–275.
9.		Marston L, Carpenter JR, Walters KR, Morris RW, Nazareth I, Petersen I. >Issues in multiple imputation of missing data for large general practice clinical databases. Pharmacoepidemiol Drug Saf. 2010;19(6):618–626.
10.		Sørensen HT, Lash TL, Rothman KJ. Beyond randomized controlled trials: a critical comparison of trials with nonrandomized studies. Hepatology. 2006;44(5):1075–1082.
11.		Freemantle N, Marston L, Walters K, Wood J, Reynolds MR, Petersen I. Making inferences on treatment effects from real world data: propensity scores, confounding by indication, and other perils for the unwary in observational research. BMJ. 2013;347:f6409.
12.		Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20(1):144–151.
13.		de Lusignan S, van Weel C. The use of routinely collected computer data for research in primary care: opportunities and challenges. Fam Pract. 2006;23(2):253–263.
14.		Hemkens LG, Benchimol EI, Langan SM, Briel M, Kasenda B, Januel J.-M, Herrett E, von Elm E. The Reporting of Studies Using Routinely Collected Health Data was often insufficient. J Clin Epidemiol. 2016 Jun 22 [Epub ahead of print].
15.		Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64(8):821–829.
16.		Collins FS, Tabak LA. Policy: NIH plans to enhance reproducibility. Nature. 2014;505(7485):612–613.
17.		Wager E, Kleinert S. Responsible research publication: international standards for authors. In: Mayer T, Steneck N, editors. Promoting Research Integrity in a Global Environment. Singapore: Imperial College Press/World Scientific Publishing; 2011:309–316.
18.		World Medical Association. WMA Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects. Fortaleza, Brazil: World Medical Association; 2013.
19.		Friedman SL. Finding treasure: Data sharing and secondary analysis in developmental science. J Appl Dev Psychol. 2007;28(5–6):384–389.
20.		Ioannidis JPA. The importance of potential studies that have not existed and registration of observational data sets. JAMA. 2012;308(6):575–576.
21.		Carter P, Laurie GT, Dixon-Woods M. The social licence for research: why care.data ran into trouble. J Med Ethics. 2015;41(5):404–409.
22.		von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Epidemiology. 2007;18(6):800–804.
23.		Langan SM, Benchimol EI, Guttmann A, et al. Setting the RECORD straight: developing a guideline for the REporting of studies Conducted using Observational Routinely collected Data. Clin Epidemiol. 2013;5:29–31.
24.		Benchimol EI, Langan S, Guttmann A. Call to RECORD: the need for complete reporting of research using routinely collected health data. J Clin Epidemiol. 2013;66(7):703–705.
25.		Benchimol EI, Smeeth L, Guttman A, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement. PLoS Med. 2015;12(10):e1001885.
26.		Nicholls SG, Quach P, von Elm E, et al. The REporting of Studies Conducted Using Observational Routinely-Collected Health Data (RECORD) statement: Methods for arriving at consensus and developing reporting guidelines. PLoS One. 2015;10(5): e0125620. Available from: http://www.record-statement.org/. Accessed July 4, 2016.
27.		RECORD [homepage on the Internet]. REporting of studies Conducted using Observational Routinely-collected Data. Available from: http://www.record-statement.org/forum/. Accessed July 21, 2016.
28.		Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:Mr000030.
29.		Sorensen AA, Wojahn RD, Manske MC, Calfee RP. Using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement to assess reporting of observational trials in hand surgery. J Hand Surg Am. 2013;38(8):1584–1589. e1582.
30.		Armstrong R, Waters E, Moore L, et al. Improving the reporting of public health intervention research: advancing TREND and CONSORT. J Public Health (Oxf). 2008;30(1):103–109.
31.		Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet.1999;354(9193):1896–1900.
32.		Prady SL, Richmond SJ, Morton VM, Macpherson H. A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials. PLoS One. 2008;3(2):e1577.
33.		Cobo E, Cortes J, Ribera JM, et al. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ. 2011;343:d6783.