Letter to the editor regarding "Clinical effectiveness and safety of powered exoskeleton-assisted walking in patients with spinal cord injury: systematic review with meta-analysis"

Dear editor

In the article “Clinical effectiveness and safety of powered exoskeleton-assisted walking in patients with spinal cord injury: systematic review with meta-analysis”, published in the March issue of Medical Devices: Evidence and Research, Miller et al¹ present a meta-analysis of the clinical effectiveness and safety of powered exoskeletons for spinal cord injury (SCI) patients. A close examination of this article shows surprising coincidences, in that two primary studies (references 25 and 33 in the reference list) report the same proportions and 95% confidence intervals (CIs) of subjects able to ambulate with an exoskeleton without assistance (Figure 2 of the study), and two different primary studies (references 26 and 28) report the same mean and 95% CIs for the distance (in meters) walked in a 6-minute walk test (Figure 4 of the study).

A likely explanation is that a single group of authors described the same patients in different publications. In fact, nine of the 14 studies included in the meta-analysis by Miller et al¹ can be assigned to three groups of studies that may contain duplicate patient information:

• New York City: references 23, 27, 32, and 34
• Philadelphia: references 25, 33, and 35
• Atlanta: references 26 and 27

Of course, these groups of researchers could have reported on a new case series in each publication, but the reported identical values for proportions and means would be surprising. Authors of literature on exoskeleton-assisted walking often list clinical and demographic information of individual subjects in a tabular format. Based on our estimates of the number of likely categories or values for gender, age, height, weight, years since injury, and level and completeness of SCI, the likelihood that Subject 1 of reference 32 is not the same person as Subject 2 of reference 27 is about 0.000015. Furthermore, sometimes, multiple individuals in two series match (eg, six of the seven subjects in reference 32 can be matched with all six subjects of reference 27). Therefore, it is clear that the chances that these studies contain information on unique individuals are infinitesimally small. Miller et al¹ should have noted the overlap in author names, investigated the uniqueness of each case series, and followed up with the investigators.

The implications of including duplicate subjects in a meta-analysis are serious; the independence of observations is violated (contravening a key assumption of inferential statistics), the CIs for characteristics of the pooled subjects are too narrow, and the heterogeneity between studies is likely to be too small.

It is not uncommon for the same study to be published twice, with the same or different primary authors. Sometimes, for a case series that is reported more than once, new subjects are added to the series, but the outcomes reported are the same^2; in other instances, somewhat different outcomes are reported for the same or largely overlapping subjects, as is the case here with the New York City studies. If the primary authors do not explicitly report individual subject characteristics, as is true for most large case series and randomized controlled trials, it is difficult for casual readers, or even systematic reviewers, to determine the degree of subject overlap. The systematic reviewer should maintain a high level of suspicion, follow up on his or her leads, and report appropriately. According to the Cochrane Handbook, “It can be difficult to detect duplicate publication, and some ‘detective work’ by the review authors may be required.”³ The Handbook suggests that the most useful pieces of information for comparing reports are author names, specific details of the interventions, numbers of participants and baseline data, and the date and duration of the study, concluding that “Where uncertainties remain after considering these and other factors, it may be necessary to correspond with the authors of the reports.”³

We strongly recommend that Miller et al¹ address our concerns and correct their report so as to remove erroneous information from the scientific literature.

Disclosure

The authors report no conflicts of interest in this communication.

References

1.		Miller LE, Zimmermann AK, Herbert WG. Clinical effectiveness and safety of powered exoskeleton-assisted walking in patients with spinal cord injury: systematic review with meta-analysis. Med Devices (Auckl). 2016;9:455–466.
2.		Creedon SD, Dijkers MP, Hinderer SR. Intrathecal baclofen for severe spasticity: a meta-analysis. Int J Rehabil Health. 1997;3:171–185.
3.		Higgins JPT, Green S, editors [homepage on the Internet]. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. Available from: www.handbook.cochrane.org. Accessed November 2, 2016.

Authors’ reply

Larry E Miller¹, Angela K Zimmermann¹, William G Herbert^1,2

¹Miller Scientific Consulting, Inc, Asheville, NC, ²Department of Human Nutrition, Foods and Exercise, Virginia Tech, Blacksburg, VA, USA

Correspondence: Larry E Miller, Miller Scientific Consulting, Inc, 1854 Hendersonville Road, 231, Asheville, NC, USA
Tel +1 828 450 1895
Email [email protected]

Dear editor

We appreciate the letter from Dijkers et al. The issue of duplicate publication in systematic reviews is important and is often difficult to identify in practice. In the current systematic review, identification of common patients was facilitated by reporting of individual patient characteristics in most included papers. Based on such data, one can uniquely identify a patient with high likelihood. That is, the chances that any two patients would exactly match on all baseline characteristics are exceedingly low. As with all systematic reviews that we perform, data are extracted to identify manuscripts that potentially reported on common patients. Based on the data extracted, we found no evidence of this occurrence. In studies performed by same author groups, we identified patient and/or study design characteristics that were distinctly different among all studies.

For example, Dijkers et al suggest that six of the seven patients in the study of Fineberg et al¹ are identical to the six patients reported by Spungen et al.² Closer inspection of the data, even when considering minor issues such as rounding, shows that none of the patients in question share identical characteristics. Specifically, all patients in the studies had different reported body weight, height, level of injury, duration of injury, age, or some combination thereof. Given this information, there was sufficient evidence to consider each patient in this systematic review unique. Therefore, the results of this systematic review and meta-analysis should be considered correct as reported. We further encourage authors who publish multiple reports from common patients to explicitly state so much in order to avoid real or perceived issues with redundant reporting in systematic reviews.

Disclosure

The authors report no conflicts of interest in this communication.

References

1.		Fineberg DB, Asselin P, Harel NY, et al. Vertical ground reaction force-based analysis of powered exoskeleton-assisted walking in persons with motor-complete paraplegia. J Spinal Cord Med. 2013;36(4):313–321.
2.		Spungen AM, Asselin PK, Fineberg DB, Kornfeld SD, Harel NY. Exoskeletal-assisted walking for persons with motor-complete paraplegia. Paper presented at: NATO Science and Technology Organization; 2013.