Toward More Robust, Clinically Decision-Supportive Evidence: Methodological Reflections on a Meta-Analysis of Acupuncture for Fibromyalgia Syndrome [Letter]

Dear editor

We read with interest the recent meta-analysis by Jin et al on the efficacy of acupuncture for Fibromyalgia Syndrome (FMS).¹ Given that complementary and alternative medicine is already recommended in clinical practice guidelines as an important component of FMS treatment,² this study contributes valuable evidence supporting the role of acupuncture in FMS symptom management. Nevertheless, several methodological and interpretative issues merit clarification.

Substantial Control Group Heterogeneity Undermines Result Interpretability

The meta-analysis pooled effect sizes across highly heterogeneous control conditions—including sham acupuncture, antidepressants, antidepressants plus exercise, moxibustion, transcutaneous electrical nerve stimulation, health education, nutraceutical treatments, and no intervention—into a single composite comparator.¹ This strategy is methodologically debatable, as these comparators differ fundamentally in intervention activity, nonspecific effects (eg, attention and expectancy), and mechanisms of action. Treating them as equivalent violates the meta-analytic assumption of between-study comparability³ and introduces extra variability.⁴ As a result, the pooled estimate becomes clinically difficult to interpret and may not reflect the true effect of acupuncture relative to any specific comparator.

Future evidence syntheses should apply more stringent criteria when selecting and pooling control groups, restricting quantitative synthesis to trials with sufficiently homogeneous comparators.³ When multiple heterogeneous comparators must be considered, network meta-analysis may be preferable, as it provides a more appropriate framework, allowing coherent integration of direct and indirect evidence and generating clinically interpretable relative effect estimates across interventions.⁵

Pooling Outcome Measures with Distinct Constructs Potentially Induces Clinical Misinterpretation

Regarding outcome selection, the authors combined data derived from multiple instruments that assess distinct constructs, without subgroup analyses by measurement tool.¹ Using pain-related outcomes as an example, scores from the VAS, NPRS, BPI, SF-MPQ, Wong-Baker FACES Pain Rating Scale, and the individual pain item of the FIQ were combined. In clinical practice, the VAS and NPRS primarily measure pain severity/intensity,⁶ the SF-MPQ covers both sensory and affective dimensions of pain,⁷ whereas the FIQ pain item focuses more on the degree to which pain interferes with function and daily activities.⁸ These conceptual differences indicate that the measures are not directly interchangeable. Even when standardized mean differences are used, pooling such outcomes risks conflating measurement characteristics with treatment effects, thereby undermining internal validity. Future meta-analyses should preferentially pool studies using identical or highly comparable instruments, while synthesizing results from conceptually distinct measures qualitatively rather than quantitatively.

English-Language Restriction Introduces Likely Language Bias

Although acupuncture originated in China and a substantial proportion of relevant randomized controlled trials (RCTs) are published in Chinese-language journals, the meta-analysis was restricted to English-language publications, explicitly excluding Chinese studies without justification for omitting Chinese databases.¹ This English-only search strategy introduces a clear risk of language bias. Methodological guidance from the JBI, Campbell Collaboration, and PRISMA Statement emphasizes the importance of comprehensive, multilingual searches to minimize selection bias.⁹ Future meta-analyses in this field should include RCTs irrespective of publication language to enhance evidence completeness and the robustness of conclusions.

Subgroup Analyses Lack Statistical Power to Explain Heterogeneity

Although subgroup analyses were conducted to explore heterogeneity, several subgroups included extremely few trials. For instance, mental health outcome was analyzed using a “female” subgroup (with no RCTs) and a “mixed-gender” subgroup (with only one RCT). Similarly, anxiety outcome was stratified by treatment frequency, with only two trials per subgroup.¹ Under such conditions, pooled estimates effectively represent individual study results and do not meaningfully improve precision or allow assessment of effect distributions. More importantly, subgroup analysis aims to test interaction effects between subgroups; when each subgroup contains ≤ 3 trials, reliable estimation of within- and between-subgroup variability is statistically infeasible. Consequently, subgroup-based conclusions (eg, statements such as “twice/week regimen contributes to anxiety improvement”)¹ lack robustness and are vulnerable to overinterpretation.

Moreover, substantial heterogeneity remained across multiple outcomes despite subgroup analyses, indicating that key sources of variability were not adequately addressed. Additional methods, such as meta-regression,⁴ are warranted to systematically examine the potential moderating effects of other variables—such as age, disease duration, baseline symptom severity, treatment dosage, and treatment modality—on the effect size.

Overly Broad Follow-Up Categorization Masks Temporal Effect Patterns

Follow-up durations were dichotomized as short-term (≤ 8 weeks) and long-term (> 8 weeks), yet “long-term” follow-up ranged from 2 to 22 months post-treatment.¹ Pooling outcomes assessed shortly after treatment cessation with those measured nearly two years later—when effects may be confounded by the natural course of the disease, other interventions, lifestyle changes, and additional factors—oversimplifies the temporal trajectory of acupuncture benefits over time. This approach likely reduces the temporal interpretative precision of the results, potentially obscuring non-linear patterns of efficacy decay, possible “rebound” phenomena, or delayed effects.

Absence of Evidence Certainty Assessment Limits Conclusions for Clinical Practice

Critically, the meta-analysis did not assess the certainty of evidence for its pooled outcomes, limiting readers’ ability to judge the overall strength and clinical reliability of the findings. For instance, in the pain and stiffness outcomes reported, if a high risk of bias or wide confidence intervals leads to downgrading the certainty to “low”, the pooled results—even if statistically significant—should be interpreted with caution in clinical contexts.¹ We therefore strongly recommend applying the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) framework,¹⁰ or the recently developed GRADE-TCM framework,¹¹ to transparently rate evidence certainty, justify downgrading decisions, and clearly distinguish conclusions supported by moderate-to-high certainty evidence from those based on low or very low certainty. Such practices would substantially enhance methodological rigor and improve the relevance of the findings for clinical practice and guideline development.

In summary, while the meta-analysis by Jin et al provides a timely synthesis of evidence for acupuncture in FMS, addressing the aforementioned methodological issues would strengthen future research. This, in turn, would generate more robust, transparent, and clinically actionable evidence to better inform decision-making regarding the role of acupuncture in FMS management.