The American Society of Pain and Neuroscience (ASPN) Practical Guidelines to Study Design and Scientific Manuscript Preparation in Neuromodulation

Introduction

Clinical trial design is a critical aspect of research in neuromodulation that serves to enhance and economize the clinical trial conduct. A well-designed clinical study based on a strong hypothesis evolved from clinical practice would facilitate the implementation of the best views of evidence-based practice. Broadcasting research findings through scientific manuscripts are essential for advancing both science and the careers of individual scientists. Although several publications provide general guidance on the writing process, few resources provide specific, pragmatic guidance for clinical researchers in neuromodulation. In light of this gap, Translational Research Committee of American Society of Pain and Neuroscience (ASPN) provides guidance on how to compose a clinical research trial and manuscript, highlighting approaches to overcome frequent challenges in the scientific research process. This article aims to describe the essential components of clinical trial design and subsequent manuscript preparation with special concentration in neuromodulation. Spinal cord stimulation (SCS) has been one of the most effective treatment modalities for chronic neuropathic pain. The initial mode of SCS was tonic stimulation. Novel wave forms (burst, high frequency), approaches (dorsal root ganglion stimulation), and techniques (pulse generator free and wireless stimulations) have been added to complement tonic stimulation as options for SCS treatment. However, the understanding of the pathways engaged in this type of treatment remains limited as there is a significant gap in knowledge of mechanism of action as opposed to clinical efficacy. To close this gap, well-designed investigative studies are needed. This will allow the field of neuromodulation to optimize outcomes and explore new indications.^1,2

The quality of the design in recently published neuromodulation clinical trials has significantly improved. The trend towards double-blinded randomized trials is very promising. However, there remains a need decrease bias and placebo effect to properly evaluate the efficacy of this treatment modality.^3,4

Clinical Trial Design

The Research Question

The research question seeks to answer an important problem by defining a hypothesis that is challenged, examined, and analyzed to provide interpretable inferences.⁵ A well-formulated research question should be precise, clinically relevant, and build on hypotheses from existing literature to generate data from patient populations of interest.

Inspiration for framing an effective clinical research question is often derived from unanswered problems in the current clinical practice, unexpected clinical outcomes, or alternative therapies. PICO (population, intervention, control, and outcomes) criteria may assist investigators by providing step-by-step guidance on the formulation of a research question and eventual initiation of a research project.

PICO Method

When designing an effective research question, investigators will inevitably consider numerous questions, which should generally be classified into background and foreground questions. The former are universal questions about a clinical issue,⁶ commonly inquiring what, when, how, and where about the disease, disorder, or treatment. A comprehensive literature review will usually provide the answers to these types of questions. In contrast, foreground questions are patient-oriented queries relating to the interpretation of a treatment or clinical condition and consideration of risk versus benefit.⁶ These sorts of clinical queries are answered by reviewing previous studies in the literature and typically comparing the different treatments.

The PICO format is considered a broadly accepted method for formulating a “foreground” research question.⁷ Sackett et al emphasized that dividing the question into four components will enable the researcher to identify the relevant information:

Population or problem

It is essential to consider the specific characteristics, demographics, and homogeneity of the study population and how similarly it represents the clinical population of interest.

Intervention or treatment of interest

The study intervention is the treatment or test to be investigated and can consist of any clinical act, including therapy, a diagnostic test, a prognostic factor, or a procedure.

Comparator or control

In order to gauge the effectiveness of the intervention, the innovative therapy should be compared to standard treatment.

Outcome

The outcome is the consequence of the intervention, such as the effectiveness of a certain procedure in controlling pain. The appropriately defined primary outcome should be easily quantifiable, specific, valid, reproducible, and relevant to the research question.⁸

Following the PICO approach requires researchers to dissect the foreground question into fragments to simplify the research question’s ultimate formulation. From the array of questions arising out of such dissection, the best one can be selected based on⁶ which question has the most significance, is most relevant to knowledge needs, and is likeliest to lead to interesting answers. In addition, one should consider the feasibility of data collection needed to assess the evidence.

As proposed by Hulley et al, a research question can be framed based on FINER (feasible, interesting, novel, ethical, and relevant) criteria⁹ and should be able to fill gaps in the current knowledge. A well-developed research question can also aid the researcher in evaluating the practicality of performing a trial; mentorship from experienced scientists can help young investigators to ensure that research effort is not needlessly wasted.⁸ To that end, the researcher should design and conduct a pilot trial, cost analysis, and consult with a statistician.

Although a clear, unbiased, and novel research question is useful for conducting successful research that engages reviewers and scientific societies, novelty in itself is not necessarily essential; a repeated question can still be valid if the study focuses on a different aspect (such as technique or patient population) of a previously explored issue. A comprehensive literature review will familiarize the researchers with previous studies to ensure that they avoid conducting a similar trial without answering a new scientific question. To maximize the utility of the literature review, Gray et al recommend following the “4S” rule, in which researchers avail themselves of systems (use of comprehensive resources), synopses (extraction of high-quality studies and abstracts), syntheses (systematic reviews), and studies (consultation of original research studies).¹⁰ A thorough literature review can assist the researcher in formulating not only the research question but also the proposal and design of the project in terms of sample-size analysis, type of statistical analysis, and current knowledge gaps in the research topic.

A focused research question leads to the systematic planning of a research project. The difficulty in framing a research question is usually not due to the lack of ideas. The challenge is to transform a novel research question into a valid study design, which necessitates refining the question. Devising a single research question that directly addresses a knowledge gap is the preferred method to produce an appropriate study design, methodology, and results.^8,11,12

Study Design

When considering the design of a study, investigators first need to identify the type of research best suited to answering a given question. Next, they must design the project with clear and standard methodology based on the chosen study type and prior studies. All the elements, including the research scope and sample-size analysis, need to be incorporated into the study design in a manner that reduces bias and avoids confounding factors. Using the population of interest identified in their research question, investigators should study a sample population with the goal of generalizing the study results to the target population. Therefore, it is critical that the sample population is accurately representative of the target population.^13,14

The next step is to do a sample-size analysis (also called a power analysis) to ascertain that the study sample size is adequate to determine the effect (if it is present) and detect the possible significant difference between the control and treatment arms.¹⁵ Then, investigators should provide a detailed description of the trial, including specifics about their inclusion/exclusion criteria (with justifications), recruitment process, study location, and type of study. Adequate argument should be provided explaining why the selected study type is suitable to answer the formulated research question. The randomization and blinding process needs to be explained with clarity and transparency. If there is any control group in the study, investigators must prepare a detailed protocol for selecting the control subjects and justify how similar they should be to the treatment group to minimize the interference of confounding factors. In addition, the methods and frequency of patient follow-up and how to manage dropouts and excluded subjects should be enumerated. Finally, the authors clear explanation of why every measurement method is used and what type of variable tests are used.¹⁶ Following all of these steps will increase the likelihood of designing an accurate, consistent, and unbiased data collection protocol that will yield reliable results.

Bias, Sampling, and Study Design

Bias in scientific research occurs when “systematic error is introduced in sampling or testing by selecting or encouraging one outcome or answer over others.”¹⁷ Biases can be classified in numerous ways, commonly based on when they occur or by the direction of change in the estimate. Below we discuss common types of bias in neuromodulation studies including selection bias, information bias, and those resulting from the study design.

One very common source of bias, selection bias, is a distortion in a measure of association due to a sample selection that does not reliably reflect the target population.^18,19 Usually this is due to an error in the method of sample collection such as a failure of proper randomization. Selection biases that readers of neuromodulation articles should pay close attention to including prevalence bias, admission-rate bias, and volunteer bias.^20,21

Prevalence, or prevalence-incidence bias, occurs when individuals with severe or mild disease are excluded, resulting in an error in the estimated association between an exposure and an outcome.²² This bias commonly affects cross-sectional and case–control studies and can be avoided by using incident cases (all individuals who change from non-disease to disease status). Admission-rate bias, commonly known as referral bias, arises when disease characteristics or exposure meaningfully differ between an intervention group and controls, resulting in the increased likelihood of unusual outcomes.²³ This can affect observational studies and may be minimized using matched controls, in which the exposed experimental group is similar to the control group. Volunteer bias, also known as self-selection bias, is a systematic error due to potential differences in those who participate and the general population.²⁴ Volunteer bias can occur at all stages of research, from recruitment to retention to follow-up. This bias can be improved by ensuring the anonymity and confidentiality of volunteers.

Information bias involves systematic error in the collection or handling of information in a study. The three main types of information biases are misclassification, ecological, and regression to the mean. Some common examples include detection bias, lead-time bias, and recall bias.

Detection bias, a misclassification bias, reflects differences in the way that groups’ outcome information is collected or the way outcomes are verified. This bias commonly affects cohort studies and can be minimized by utilizing blind assessors, proper randomization, and statistical adjustment for perceived differences. Lead-time bias occurs when early diagnosis of a disease and subsequent possible prolonged survival may not represent the real-life targeted population outcome. This bias can commonly affect screening studies and may be minimized with a more diverse study population. Recall bias refers to a systematic error that occurs when participants do not remember previous events or experiences accurately or omit details.^25,26 This can affect all studies, particularly those dependent on self-reporting. This can be minimized by obtaining information in a timely manner and factoring this tendency in when selecting the questions, collection method, and study design.

To critically assess and minimize the potential of bias, a thorough understanding of study design is essential. Study design can be broadly categorized as observational or interventional. Observational study designs, also called epidemiologic study designs, are often retrospective and are used to assess potential causation in exposure–outcome relationships and therefore influence preventive methods. Interventional studies are often prospective and are specifically tailored to evaluate the direct impacts of treatment or preventive measures on disease.²⁵ Examples of specific common study designs include case-control, cohort, cross-sectional, interventional, and randomized control.

Case-control: Case–control studies are observational and are designed to estimate the odds of some outcome. As a result, such studies are at an increased risk for bias, particularly recall bias. When evaluating case–control studies, both readers and investigators should review the appropriateness of the controls, timing of exposure to outcome, blinding of the review, and selection biases.

Cohort: To limit the influence of confounding variables in cohort studies, the investigators can compare matched subjects or similar groups. However, due to the lack of randomization, an imbalance of characteristics can exist. When evaluating cohort studies, researchers must assess the defined outcomes as well as the extent to which the study cohorts accurately represent the population of interest. Additionally, the number of subjects and the duration of study follow-up should be comparable across cohorts.

Cross-sectional: Random sampling is paramount in cross-sectional studies as selection bias may result in a flawed measure of prevalence and calculation of risk. Though the exposure and outcome at a single time can be assessed, temporality cannot be demonstrated. When one is evaluating cross-sectional studies, the response rate as well as validity and reliability of the reported data should be carefully evaluated based on the sample selection as well as limitations of the study design.

Interventional: Interventional studies, also called experimental studies, provide the opportunity to measure the impact of an intervention on some outcome. When evaluating interventional studies in general, one should consider the recruitment of the subjects, assignment of the groups, and overall selection.

Randomized Control: Randomized controlled studies compare participants who have been allocated to treatment or control groups using random assignment. It is important that these studies are conducted without confounders (variables that are associated with or have a relationship with both the intervention and the outcome of interest).

Other techniques, such as concealing allocation, blinding, analyzing results by intention to treat, and measuring compliance, are used in randomized controlled studies to further strengthen the results of the intervention and the study design. Further, the target population, sample size, and power calculations (the likelihood that a study will find a significant result) should be clearly accessible to the reader.

Outcome Factors and Study Factors

Outcome Factors:

1) Are all relevant outcomes assessed?

The Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) identified six core outcome domains that should be taken into account when designing a clinical trial related to chronic pain. The domains include (1) pain, (2) physical functioning, (3) emotional functioning, (4) participant ratings of improvement and satisfaction, (5) symptoms and adverse events, and (6) participant disposition.²⁷ The work of the IMMPACT-II group, which further characterizes core outcome measures for chronic-pain studies.²⁸

The above outcome measures supported by the IMMPACT series of publications provide a solid checklist which can be used to evaluate pain-related neuromodulation studies. As neuromodulation literature has shown favorable trends in decreasing opioid consumption and outcomes associated with the high utilization of systemic opioids,²⁹ opioid utilization or Morphine Equivalent Dosage (MED) should be considered as a standard outcome measure in neuromodulation studies. Measurement of adverse events should likely also be both actively and passively captured in neuromodulation-specific studies as several adverse events, such as device malfunction and stimulator electrode migration or fracture, are difficult for patients to recognize symptomatically. Active capture of these variables is essential for gaining a comprehensive understanding of the risk profile of any new technology that is introduced. Health economic data are another outcome measure of increased interest, given rising healthcare costs and the positioning of neuromodulation therapies as potentially cost-effective compared to the conservative management of chronic pain. Finally, efficacy of treatment will need to continue to evolve beyond primary endpoints at 6 months or less. Outcomes at 12 months and beyond are likely needed in clinical studies evaluating invasive implantable devices to evaluate the long-term clinical success needed to translate study results to clinical practice. In summary, we recommend that neuromodulation publications are assessed for opioid utilization, active measurement of adverse effects which may go undetected by patients, health economic data, and efficacy of treatment at 12 months and beyond.

Data Analysis

Thorough and appropriate evaluation of data is essential to the assessment of clinical neuromodulation research. Critical review of a study’s data analysis can start with clear identification of the study’s reported main findings. In regard to patient safety, assessment of a study’s data analysis should start with evaluating the extent to which both the number and nature of adverse events have been addressed. The properties of the collected data, in combination with the questions asked, can be used to confirm the appropriateness of the statistical tests chosen. Confounding factors must also be carefully considered, especially if multiple endpoints have been collected. The 2008 Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) recommendations provide a review of various criteria, adjustments, and analyses for determining potential confounding factors and each given study’s options for their prevention and/or mitigation. This is particularly important given the increased risk of Type I error associated with multiple endpoints in the study of pain treatments.³⁰

Key Statistical Concepts and Common Pitfalls

Statistical tests are routinely used to compare outcome measures across groups or before and after some intervention. However, a large proportion of medical publications contain statistical errors or inappropriately use statistical methods that may lead to incorrect interpretations and misleading conclusions.^31–34 Being aware of common errors can help researchers avoid common problems and help readers to critically review neuromodulation manuscripts.

The most important statistical considerations should occur during the study-design phase, as any errors at this stage carry forward and can negatively affect the validity of results. It is critical that a study describe methods for choosing sample size, blinding, randomizing, and conducting intention-to-treat analyses. When interpreting statistical tests, investigators must check assumptions regarding underlying statistical methods, such as confirming that variables conform to a normal distribution when using parametric statistics (such as the t-test or ANOVA).³² Most studies never verify such assumptions, leading to inconclusive analyses. Fortunately, some recent neuromodulation trials have adhered to these standards.^35,36

When reporting differences in outcome measures between groups, it is critical that the study report at least 3 quantities: 1) the absolute difference between groups, (2) a measure of precision of the estimate (eg, confidence interval), and (3) the probability that observed values may be due to chance alone (p-value). These steps give the reader closer access to raw data and facilitate interpretation of results in clinical context. The precision of some estimates (eg, mean pain score) is best represented as a 95% confidence interval (CI), in which 95% of the interval estimates can be expected to contain the true population value.³³ Therefore, if some intervention is associated with an estimated average pain-score reduction of 25% with a 95% CI range from 5% to 45%, it is possible that the true pain-score reduction may be only 5% (or 6%, 44%, etc.). The CI gives insight into both the accuracy and precision of the estimate.

Interpretation of p-values in the medical literature is fraught with errors and common misconceptions.^33,37,38 Conventionally, the type I (false positive) error rate for comparing group means is set at 5% (alpha = 0.05). It is critical to understand what a p-value is and what it is not. The correct interpretation of the p-value is the probability that observed values (eg, sample mean) occur solely by chance. Importantly, the p-value is not the probability that observed effects are due to chance nor the probability that the null hypothesis is true (which instead is the “false discovery rate”).^32,33 The actual false discovery rate associated with a p-value of 0.05 is at least 23% and typically higher, so no decision should be based on a single p-value or study.³⁹ Further, smaller p-values do not suggest higher significance; a pain reduction of 1% could be very statistically significant (p<0.001) if sufficient samples are collected. Next, the lack of a statistically significant difference does not equal similarity between groups: “The absence of evidence is not evidence of absence “.⁴⁰ Most neuromodulation trials do not adequately correct for multiple comparisons,^4,35,41,42 confounding secondary outcome results. Finally, the magnitude of the p-value reveals nothing about the clinical significance of a result, which must be inferred in the context of meaningfulness of the data being collected.

Clinical significance refers to the established difference in a particular outcome variable that must be achieved to reflect a clinically meaningful or useful result. The Minimal clinically important difference (MCID) for a measurement reflects the empirical real-world differences that must be obtained to reflect a practical change in some outcome. The MCID for various common instruments has been established and should be referenced when interpreting reported differences between groups or interventions in clinical studies.⁴³ Because neuromodulation and other device trials include fewer subjects than drug trials, it is critical that multiple clinically meaningful outcomes surpass statistical thresholds for sound inferences to be made. Interpretation of group differences in such trials follows a distinct approach from individual differences, and care should be taken not to conflate the two.²⁸ By design, standard randomized clinical trials (RCT) seek to identify group-wide differences while minimizing individual heterogeneity; therefore, it is often problematic to apply the results from RCTs to any real-world single patient with more complex circumstances.

When designing a test statistic that will guide decision-making, researchers must be able to describe the relationship between those who may test positive (ie, test indicates the presence of disease) with those who are truly positive (ie, disease is actually present). The most common way to represent test performance is through receiver operating characteristic (ROC) curves. Depending on the purpose of the test, investigators may choose to maximize sensitivity (eg, screening tests that will not miss true positives), maximize specificity (eg, confirmatory tests that will not miss true negatives), or balance the two. Concepts such as positive predictive value (PPV) and negative predictive value (NPV) further relate the proportion of subjects with either a positive or negative test result to those who truly have or lack the condition, respectively.

Finally, when considering the results of a clinical study, investigators must carefully interpret measures of treatment-effect size; such measures help to conceptually bridge statistical results with real-world concerns. Relative risk (RR) indicates the odds that an intervention will lead to a given effect; it can be calculated by dividing the proportion of subjects who have a particular outcome (eg, improved pain) in the experimental group (Ep) by the control group (Cp). To determine the absolute risk reduction (ARR) provided by the intervention, Cp is subtracted from Ep. To frame this value in relative terms, ARR can be divided by the Cp again to achieve the relative risk reduction (RRR). Alternatively, by taking the reciprocal of the ARR, researchers can define the number of subjects that must be treated with an intervention before one subject actually gets a benefit (number needed to treat, NNT).

Interpreting Conclusions

The appropriate use of statistical methods on various outcome measures should provide converging evidence to support each conclusion. While p-values and effect sizes can assist in making inferences about results, the underlying biological plausibility should be used to bridge statistics with conclusions of a study.²⁸ In general, it is important to evaluate the absolute effect sizes and compare them to other studies to verify external validity. Authors should be transparent about the limitations of their analyses and relate their findings to previous conclusions from related and applicable studies. Specifically, results that fail to replicate prior studies should be explained, and confounding factors should be described.

It is impossible to conduct a clinical study without the influence of bias, and authors should disclose all conflicts of interest or ethical issues (eg, financial, personal) in the article. References cited should strive to include impactful older studies of relevance as well as recent reports closely related to the subject matter. Ultimately, statistical methods and tools of inference are only as valid as investigators’ attempts to maintain clarity and access to the raw data as closely as possible. It remains the responsibility of both the writer and the reader to make sound conceptual connections between the stated intention of the experiments and conclusions drawn.

Trial Types

What Type of Study to Design?

Regarding trial design, researchers and readers must clearly understand the difference between efficacy and effectiveness. Efficacy is established by comparing a treatment to a placebo or sham control, while effectiveness is evaluated by comparing either two different treatments or a single treatment to standard-of-care.⁴⁴ For example, Fischgrund et al randomized patients to basivertebral nerve ablation versus a sham procedure, thereby evaluating the efficacy of basivertebral nerve ablation,⁴⁵ while Khalil et al evaluated basivertebral nerve ablation against standard-of-care, thereby evaluating its effectiveness.⁴⁶ Most neuromodulation randomized controlled trials evaluate the comparative effectiveness of one neuromodulation treatment against either another neuromodulation treatment (such as one waveform against another waveform,^41,42 dorsal root ganglion stimulation against traditional spinal cord stimulation,³⁵ or spinal cord stimulation again reoperation)⁴⁷ or against conventional medical management.⁴⁸ The placebo influence includes patient enthusiasm and optimism to have a new treatment, increased doctor–patient interactions, increased expectations of treatment effects, and decreased negative emotions such as anxiety and invasiveness of the procedure.^49–51

The placebo effect in interventional procedures with medical device insertion can be significant, particularly in measuring subjective outcomes like pain.^52–55

There is a controversy surrounding placebo (sometimes called sham intervention) methodology as it may be challenging to reproduce the entire experience of the treatment arm without breaking patient and physician blinding. In addition, the risks of patients receiving an invasive procedure without active treatment raise ethical concerns. A common objection is that the inclusion of placebo controls in surgical trials poses ethical problems because it involves risks for the patient or may not offer the best possible treatment.⁴⁰ There is an ongoing debate on appropriate methods to control and evaluate treatment efficacy and when and how the placebo control should be implemented. It is a challenge to create appropriate placebo control conditions in neuromodulation studies, which ideally imitate all elements of the treatment except the active component.^56,57 While there are ethical considerations, there are also risks in not validating interventional procedures in placebo-controlled trials because patients might undergo invasive and risky treatments without proven efficacy.^56,58

A recent review by Duarte et al⁵⁹ discusses the use of such trials to evaluate the efficacy of SCS and addresses the methodological concerns in these studies. They highlight the importance of the transparent reporting of stimulation programming parameters, patient position during perception threshold measurement, management of the patient handheld programmer, frequency of recharging, and assessment of the fidelity of blinding by pain researchers.⁴⁶

Researchers must also decide if they wish to design an explanatory study, which uses strict inclusion criteria and seeks to enroll “ideal” patients to test a hypothesis, or a pragmatic study, with “real-world” patients, to guide policy. Readers need to understand the difference and take this into account when interpreting the data from a publication. Explanatory studies of medial branch radiofrequency ablation have often used strict criteria and 2 or 3 diagnostic blocks with higher percent relief requirements for inclusion, and then randomized patients to radiofrequency ablation versus sham radiofrequency ablation.^60–63 These studies were designed to assess the efficacy of medial branch radiofrequency ablation for facetogenic pain. The MINT study – a pragmatic study with permissive inclusion criteria and single diagnostic blocks with 50% response for inclusion – compared radiofrequency ablation to exercise therapy. This study was designed to guide policy in the Netherlands.⁶³ However, its permissive “pragmatic” inclusion criteria resulted in a negative trial that has been used to argue against payment for this treatment.^64,65

Researchers should consider where the evidence is for a given intervention to decide what type of study is needed next to raise the evidence to the next level. The US Preventive Services Task Force (USPSTF) provides a Hierarchy of Studies, grading the evidence for a therapy as Evidence levels I–III based on the design of the studies supporting the therapy’s use.⁶⁶

 Of note, this Hierarchy of Studies does not specify whether placebo- or sham-controlled studies, comparative effectiveness studies, or explanatory or pragmatic studies are needed to claim “Level I” evidence for a therapy. The USPSTF also provides a Level of Certainty rating of high, medium, or low, based on the evidence strength and a degree of recommendations of A-D or I (insufficient, low-quality, or contradictory evidence).⁶⁷ Pain researchers and fundraisers should consider these levels when deciding where the evidence is and which types of trials to design or fund for a given treatment at a given time.

Minimizing Bias in Your Study Design

Minimizing bias is one of the foundations of high-quality research. A popular tool for evaluating the quality of evidence is GRADE, or Grading of Recommendations, Assessment, Development, and Evaluations. An overall GRADE quality rating can be applied to a body of evidence, with the lowest quality of evidence from the critical decision-making outcomes data being selected.⁶⁸ Using this system, both high- and low-quality evidence can be rated down if it comes from studies that suffer from a high risk of bias.^69,70 One of the top reasons for rating down the quality of evidence is study limitations or risk of bias. According to the modified Cochrane Collaboration tool to assess the risk of bias, bias is assessed as a judgment (high, low, or unclear) for individual domains (selection, performance, attrition, reporting, and other). Bias is common. In fact, about half of all trials are not published, and a third of all trials have outcomes that are omitted due to bias; and trials with statistically significant results are twice as likely to be published as trials without statistically significant results.⁷¹ The following are the seven evidence-based domains that are fundamental to the risk of bias in research.⁷²

1. Random sequence generation

Random sequence generation aims to reduce selection bias and can best be described as the method that is used to generate the allocation sequence of subjects in a trial. In designing a study, one should generate the allocation sequence to produce comparable groups of subjects in the intervention and control populations to minimize bias. It is also important to describe the method used in enough detail that an outside reviewer can independently assess whether subsequent trials should produce comparable groups. Inadequate randomization sequence correlates with a high risk of selection bias; conversely, if the random sequence generation does produce comparable groups, there is a low risk of selection bias. If the sequence generation is not described in enough detail, the risk of bias is unclear and will be judged as such by an independent reviewer.

(b) Allocation concealment

Allocation concealment is another method used to reduce selection bias. Allocation concealment is the method used to blind both practitioners and study participants to the allocation sequence so that enrollment in the intervention group could not have been foreseen during enrollment in the trial. Inadequate concealment of allocations prior to the assignment risks selection bias, with certain patients being selected preferentially for treatment or for the control arm. If intervention allocations could not have been foreseen before or during enrollment, there is a low risk of selection bias. If this is not described in enough detail, the risk of bias is unclear. In designing a study, one should not only conceal the allocation sequence but also describe the method of concealment in sufficient detail to allow an outside reviewer to determine if the intervention allocations could have been anticipated before or during enrollment.

(c) Blinding of participants and personnel

The blinding of participants and personnel is an important tool to minimize performance bias. Performance bias may result from participants and personnel involved in the study having knowledge of the allocated interventions or treatment. However, if blinding is effective, there will be a reduced risk of bias. One must describe all measures used, if any, to blind study participants and personnel from knowing which intervention a participant received. Information should also be provided regarding the effectiveness of blinding efforts. If the study does not describe the intended blinding measures and the effectiveness thereof, there is an unclear risk of bias, and the study will be reviewed accordingly.

(d) Blinding of outcome assessment

The blinding of outcome assessment refers to the blinding of the outcome assessors from learning which intervention a participant received, an important step in minimizing detection bias. The study should include all measures used to blind the assessors as well as an explanation of whether or not the intended blinding was effective. If the measures are well described and the intended blinding was effective, there is a low risk of bias. If the outcome assessors are informed about the intervention, there is a high risk of bias. If this is not described in detail, the risk of bias will be deemed unclear.

(e) Incomplete outcome data

The complete outcome data for each of the main outcomes measured in a study must be included to minimize the risk of attrition bias. This includes attrition and exclusions from the analysis. The study should state whether attrition and exclusions were reported, as well as the numbers in each intervention group and the reasons for attrition and exclusions. There is a high risk of bias with incomplete outcome data; however, if the handling of incomplete data was appropriate and described in detail, it will reduce the likelihood of producing bias. Insufficient reporting of attrition and exclusions will lead a reviewer to an unclear assessment of bias.

(f) Selective reporting

Selective reporting occurs when findings such as intervention efficacies that are statistically non-significant are not reported or are only partially reported, which leads to reporting bias. There may also be bias in selection of the reported result, wherein authors perform multiple analyses for a particular outcome and only report the result that yielded the most favorable effect.⁷³ Selective outcome reporting has a high risk of bias. It is very common for insufficient details of studies in this category to permit judgment by a reviewer.

(g) Other Bias

If there are any concerns about bias that are not mentioned or covered in the above categories, they should be presented in detail in the article for review.

How to Report Your Trial

Prospective studies are expected, and in some situations required, to have their design registered before their outset with registries such as the US National Library of Medicine’s ClinicalTrials.gov (https://clinicaltrials.gov/), the EU Clinical Trials Register (https://www.clinicaltrialsregister.eu/), and Australian Clinical Trials (https://www.australianclinicaltrials.gov.au/). These registries were created for many reasons, such as ensuring research integrity and reducing publication bias. For example, reporting a study’s planned primary outcome on such a registry prior to beginning the study provides assurance that the primary outcome was not changed after data collection.⁷⁴

When writing the manuscript for a randomized controlled trial, the CONSORT (Consolidated Standards of Reporting Trials) Statement guidelines should be followed.⁷⁵ These include a 25-item checklist with a flow diagram that shows the numbers of patients who will be assessed for eligibility, randomized, allocated, lost to follow-up, and analyzed.

Manuscript Preparation

Overview of Paper

Writing a manuscript requires a significant amount of effort. Therefore, it is essential to consider multiple aspects before embarking on such a lengthy endeavor. Future peer evaluation is one such crucial consideration. Another is the specific criteria that will weigh the value of the paper according to current clinical guidelines. In this section, we will provide you with the relevant elements most vital to prepare your manuscript, one of which is the most clinical relevance to the performed research. The formulation of an effective hypothesis necessitates a consideration of its value for other clinicians over a particular aspect of your practice that you may wish to explore. One way to determine the potential relevance of your study is by performing a thorough review of the literature to confirm that the subject of the study has not been previously evaluated and to assess whether or not your research will add valuable information to the topic in question.

a. Article title: Once your paper is published, the title will significantly determine how the manuscript will be indexed, making it an essential aspect of your work. If it reflects the content of the paper accurately, it will facilitate the manuscript’s identification by other authors and reviewers within the electronic database. The title should reflect the nature of the study (ie, case report, prospective cohort study, retrospective analysis, randomized control trial, etc.). In addition, it should summarize the research conclusions and raise readers’ awareness of the content of the study. The title and keywords should attract others in future literature searches. Thus, the title is possibly one of the most critical portions of the manuscript and has to provide an accurate overview of the study’s content.

b. Publishing journal: Understanding that specific journals will not be open to accepting particular types of publications is essential. Certain journals may not consider publishing a case report or a retrospective analysis because of their limited value and potential bias. Remaining conscious of the journal scope and guidelines for authors is a must, especially when striving for publication in highly reputable journals.

c. Authors and their institutions: In the current climate of the medical profession, with its schemes of values and rewards, authors must disclose any potential conflict(s) of interest that may have biased the execution or interpretation of the research. All the authors must be involved in the editing process and responsible for the manuscript’s content and discussion. The need for transparency is so critical that most journals go beyond requesting the disclosure of the personal conflicts of the authors alone, broadening the scope of disclosure to include the authors’ close family interests. Potential conflicts include, but are not limited to, speaking engagements, consulting fees, participation in scientific advisory boards, royalties, and ownership of stocks in companies that may benefit from the execution and message of the manuscript.

Abstract

The abstract is a structured, clear, and unbiased summary of the study design, methods, results, and conclusions. Because some readers use the abstract to determine the value of reading the full manuscript⁷⁶ and the majority of readers rarely venture beyond the abstract and assess the trial based solely on the information provided therein, the relevance of this section cannot be overemphasized. Although the journal regulations may constrain the number of words, the abstract should include sufficient information regarding the trial. If properly presented, the abstract will provide a quick tool to assess the relevance of the findings and reflect the content of the full manuscript. Claims included in the abstract should only present information that appears in the paper.^77,78 The statement on the CONSORT website provides a list of the critical aspects to be reported in an abstract.⁷⁹

The section starts with a brief introduction, contextualizing the study within its field and describing the intervention the study will test. Then, the authors should describe the study design: Is it a case series, a prospective feasibility study, a prospective cohort study, a retrospective study, or a randomized control trial? Was the study parallel, cluster, non-inferiority? What measuring tools were used to assess the primary outcome? Finally, the hypothesis or aim of the study should be clearly stated in the abstract’s introduction to help readers understand the context of the manuscript.

The next section of the abstract, which focuses on the materials and methods, starts by describing the participants, including the eligibility criteria and the setting where the data were collected (eg, tertiary health center, rural area, private practice, etc.). This section should describe the type of intervention used to measure outcomes in each group and provide a clear definition of the primary outcome. If randomization was used, the methods of the subjects’ allocation should be described. In the case of a blind study, the authors should describe who was blinded (single-blinded, double-blinded) within the study design.

The results section should report the number of participants, what the recruitment status of the trial is, the number of subjects analyzed in each group, the results for each group in relation to the primary outcome (with estimated effect size and precision), and any critical adverse events.

Finally, the conclusion section should provide a general interpretation of the results. Most journals currently request the trial registration name, the number of the study, and any funding sources. All of this must be accomplished within any structural constraints or word limits that given journals may impose.

Introduction

The introduction serves to justify the value of the research by placing the purpose of the evaluated intervention within the context of any previous clinical trials or the lack thereof. In explanatory or pragmatic terms, authors may explain the background of the problem, the magnitude of the situation, and the effect of other interventions. It is a good idea to describe any evidence of the benefits and/or adverse effects of the planned intervention, as well as to provide a potential explanation of how it may work. Perhaps the most important aspect of the introduction is the clear elucidation of the authors’ objectives and/or hypothesis. In differentiating between the objectives and hypothesis, it should be kept in mind that the hypothesis is more specific and requires specific statistical evaluation.

Methods and Materials

The first part of the methods section should specify the design of the study.

In the case of retrospective or prospective studies, the authors should provide a clear description of the data collection and stipulate whether patients were consecutive or not. Descriptions of randomized control trials (RCTs) should detail the conceptual design (eg, whether it is a superiority or non-inferiority design, whether it is crossover or cluster, the number of arms, etc.). Authors should also clarify the device allocation ratio and whether an investigator device exception was required, and any unpredicted situations necessitating changes to the protocol (such as a slow enrollment rate and/or funding issues) should be noted when pharmacological agents are involved. On certain occasions, RCTs use adaptive design protocols to allow modifications in the sample size or the number of arms.

The methods section should also include the eligibility criteria, methods of recruitment, use of informed consent, IRB approval, and report of the study to clinicaltrials.com. Additionally, the authors should describe the research setting (eg, hospital, ambulatory center, office practice, etc.) and the number of sites to enable readers to determine whether the results of the study are applicable to them. There should also be a detailed description of the intervention in the control and test arms; in cases where the control arm is going to receive standard-of-care or comprehensive medical treatment, it is essential to define the authors’ interpretation of this definition.

The primary outcome, which is used to calculate sample size, is the pre-selected endpoint that is the most relevant for the authors, patients, funding agencies, policymakers, and insurance carriers. Other outcomes of interest may be recorded as secondary endpoints. Due to their subjectivity, some outcomes, such as those in the field of pain, are considered controversial. Over the last decade, it has become common to use multiple patient-reported outcomes to evaluate the complexity of pain perception. However, healthcare providers are increasingly using other tools to assess patient outcomes, rather than relying solely on pain scores; these tools have been utilized in clinical and research fields to examine physical function, quality of life, level of depression, and opioid consumption for a better understanding of treatment outcomes. The corresponding increase in assessment tools absolutely necessitates the critical evaluation of the appropriate, controlling multiple comparisons and clinically validated scales. For example, the Oswestry Disability Index is specific for low back pain; using it in patients with neck pain will invalidate all related results.

Based on the estimated outcomes in each evaluated group, the authors should calculate the sample size. In general, the sample-size population is inversely related to the expected difference between the groups. In determining whether the final results reached statistical power or not, it is helpful if the authors include the details of how attrition or lack of compliance was incorporated in the size calculation. In certain situations, an interim analysis is performed to determine if the study should be stopped or whether recruitment should be continued beyond the initially calculated sample size. Occasionally, poor results may lead to stopping the investigation. An independent data monitoring committee is preferable for performing interim analysis to reduce the risk of premature study conclusion.

When one is performing an RCT, the method used for the random allocation sequence has to be presented to facilitate the external evaluation of potential bias. The primary purpose of the randomization process is to ensure similar numbers of patients and similar variable distribution between groups. The authors have to describe the mechanisms employed to maintain that allocation and its concealment. Conventional techniques involve the use of external parties (like the pharmacy), a central telephone randomization system, or numbered identical opaque containers or envelopes; however, recently, computer-based allocation systems have become more common.

Patients may respond differently if they are aware of which group they were assigned. Similarly, un-blinded healthcare providers may influence the subjects, based on their own bias regarding the intervention. Studies that include blinding should indicate all blinded populations (eg, patients, healthcare providers, clinical coordinators). Blinding is sometimes hard to incorporate in neuromodulation studies, especially when conventional, paresthesia-based stimulation is used as a control. The same consideration applies to control arms using comprehensive medical or standard therapy. Another results-related consideration when designing a trial is the timepoints for scheduled follow-ups; in the field of neuromodulation, the follow-up period is typically within the window of 12 to 24 months, but this may vary depending on the type of treatment being investigated.

The statistical method employed for each analysis has to be clearly presented and should include enough details to allow those versed in statistics to confirm the results. In the description of the statistical method, the authors should mention the confidence interval for the estimated effect and the statistical significance by p-value to determine the probability that observed VALUES in a group (with respect to another group) are the result of chance.

The authors should also mention whether intention-to-treat (ITT) or per-protocol analysis was used. ITT analysis, in which all patients in a randomized group are considered for the final statistical analysis, is used to avoid bias. Factors that may affect this analysis include attrition and protocol deviations. In this case, missing data may be imputed based on other collected information. One method often used is the last observation carried forward. Although it is a simple method, it may create bias by assuming that at a different timepoint, the outcome was similar to a previous point.⁸⁰ Recently the use of modified ITT (mITT) has become more popular. This analysis excludes patients who failed to adhere to the protocol. In contrast to the per-protocol analysis, only those patients that received the treatment would be evaluated. In this case, the study should be properly labeled as a non-randomized observational study. In terms of the secondary-outcome analysis, it is essential to consider that the sample size was not calculated to achieve superiority for these outcomes and is therefore generally discouraged.

Results

Although authors have a tendency to express their data assessment in this section, the objective should be to present the results of the study so that the well-versed reader may make his or her own conclusions. It is important to present the attrition rate and the reasons for the loss of follow-up. In almost any study, the loss of patients for follow-up is unavoidable and has to be differentiated from those patients excluded from the study for other reasons. Failure to report the different causes for the loss of follow-up may represent an inherent bias in the study. In ideal circumstances, the section should detail the individuals screened for enrollment, the participants randomized to each study group, the number of those allocated to a group that received the intended treatment, and the number of patients in each group who completed the follow-up visits up to the primary outcome. In summary, the section should report the number of participants included in the primary analysis.

The value of these considerations is evident when considering the use of ITT to analyze the data. Imputation analysis may be used in particular circumstances to fill the information gaps created by the loss of captured data but only if other collected information allows for the use of this tool. In cases in which authors decide to use this tool, a strong argument has to be generated to justify this approach.⁸¹ Patients who were lost for follow-up or who did not meet the eligibility criteria should be reported in the flow diagram as protocol deviations, with an appropriate description of the reason for the exclusion of the participant.

The duration of enrollment, length of follow-up, and patient baseline data, including demographics, should be reported either as part of the narrative or in a table.

For primary and secondary outcomes, the authors should note the confidence interval in addition to the p-value. In studies in which the statistical significance is not obtained, the description of the magnitude of the confidence interval is essential as it indicates that under the evaluated conditions, the clinical significance of the difference between the groups may not be totally ruled out.

Finally, adverse events and serious adverse events, including recurring events, and the number of participants that abandoned the study due to these events have to be reported for obvious reasons. A description of what triggered an adverse event should be available for review, for those assessing the safety of the studied therapy.

Discussion

Quite often, authors use this section to describe the importance of their findings, with an extensive narrative and bibliographic references to support them. To prevent this tendency, certain journals recommend limiting the section to a summary of the findings, a description of potential explanations to justify results, a brief assessment of pertinent literature comparing the outcomes to previous results, a description of the limitations of the study, and a summary of the clinical implications. Of particular importance is the description of potential limitations, which should be focused on weaknesses that may be important for readers to understand and for future researchers to avoid. A potential limitation refers to the uniqueness of neuromodulation studies, in which the therapy is expected to provide 50% pain relief to be considered significant. Consequently, reporting the clinical significance relative to statistical significance is crucial since other therapies, including pharmacological interventions, are gauged differently. At the end of the discussion, it is valuable to analyze whether the study was conducted appropriately to determine its internal validity and, assuming that no significant issues were found, conclude that the results could be generalized to larger populations (external validity).

Conflicts of Interest and Professional Medical Writers

Authors should disclose their pertinent conflicts of interest and any use of professional medical writers. Before deciding to use the assistance of professional medical writers, the authors should consider that some journals may have policies against their use, and some journal editors and peer reviewers may frown upon their use due to a concern that professional medical writers may be more likely to overstate the results of a study, especially when writers are paid to write the manuscript by a pharmaceutical company or device manufacturer.

Conclusion

Clinical research should provide evidence that is both valid and relevant to physicians’ patients and practices. Clinicians should be aware of a systematic approach to design research trials and effectively report their results. This article provides the guidance to the design of studies as well as preparing organized and transparent manuscripts in neuromodulation to assist physician scientists in this dynamic field. Well-designed studies, when done and reported effectively, support evidence-based medicine and foster improved patient outcomes.