Quality of the Evidence Supporting the Role of Acupuncture Interventions for Vascular Dementia

Introduction

Vascular dementia (VaD) is a type of dementia caused by the clinical or subclinical cerebrovascular injury resulting from ischemic/hemorrhagic stroke or other cerebrovascular diseases.¹ It is clinically characterized by impairment of neurological positioning, accompanied by difficulties with intelligence, calculation, orientation, emotion, memory, and behavior.² As the second most common type of dementia, after Alzheimer’s disease, VaD afflicts nearly 46.8 million individuals around the world. In Europe and North America, VaD accounts for about 15% to 20% of people with dementia, while in developing countries this proportion has risen to approximately 30%,³ and cases are expected to increase as the population ages.^4,5 Besides vascular problems, risk factors such as smoking, age, alcoholism, and diabetes should not be ignored.⁶ Currently used primary preventive drugs, including cholinesterase inhibitors and excitatory amino acid receptor antagonists, have very limited efficacy against vascular cognitive impairment,⁷ and are associated with significant side effects including gastrointestinal discomfort, vomiting, irritability, and dizziness.⁸ Therefore, the search for effective treatments for VaD remains an urgent issue.

Acupuncture has been used in China for more than 2000 years as an economical treatment with few side effects. Acupuncture is gradually being used as an adjunctive treatment for VaD. A study shows that electroacupuncture can improve memory and spatial learning via the miR-81/IL-16/PSD-95 pathway in VaD rats.⁹

The number of studies and SRs/MAs reporting the effectiveness and safety of acupuncture for VaD is increasing. As the very top of the evidence pyramid, SRs/MAs are generally considered helpful in identifying, evaluating, and synthesizing research-based evidence to assist clinical decision-making.¹⁰ However, due to the various potential risks of bias in the evidence formation process, it remains questionable whether evidence from SRs/MAs can provide decision guidance to clinical staff.¹¹ The overview is to systematically integrate current evidence by assessing quality and outcome indicators for inclusion in SRs/MAs to provide guidance for clinical decision-making and to identify critical gaps in the use of evidence.¹² Therefore, our research used an overview approach to critically assess and scientifically report the quality of SRs/MAs in acupuncture for VaD.

Methods

The currently overview written research protocol has been published in the INPLASY database under the registration number: INPLASY202280106. The methodology of this study followed the Cochrane Handbook guidelines as well as some high-quality overviews.^13,14

Search Strategy

Two researchers (SHS and ZXC) independently searched the Cochrane Library, PubMed, Embase, China National Knowledge Infrastructure (CNKI), China Wanfang Database, VIP Journal Database and China Biomedical Literature Database from the establishment of these databases to August 9, 2022. No language restrictions were set. We used the search method of Medical Subject Headings (Mesh) words combined with free words to search, Mesh words include, “Acupuncture”, “Vascular Dementia”, “Systematic Review”, and “Meta Analyses”. Additional references of identified literature or relevant quotes from experts were manually checked for possible missing literature. The complete search strategy for this study was provided in Supplementary File 1.

Eligibility Criteria

Literature Inclusion Criteria

1. Type of study: Based on SRs/MAs of the RCT on the topic of acupuncture for VaD.
2. Type of participant: Participants diagnosed with VaD by any of the internationally recognized diagnostic criteria. Diagnostic criteria were as follows: Diagnostic and Statistical Manual of Mental Disorders or International Classification of Diseases,¹⁵ Chinese Classification of Mental Disorders,¹⁶ Alzheimer’s Disease Diagnosis and Treatment Center criteria,¹⁷ or National Institute of Neurological Disorders and Stroke and International Association for Neuroscience Research.¹⁸
3. Type of intervention and comparator: The comparator group consisted of placebo or drug therapy (DT) such as Chinese herbal medicine, cholinesterase inhibitors, memantine, psychostimulants and nootropics, or drugs with vasodilatory effects. The intervention group received acupuncture treatment, or the intervention group added acupuncture on the basis of the control group.
4. Type of outcome: Outcomes assessed in this study included NIH stroke scale (NIHSS), Mini-Mental State Examination scores (MMSE), Scale for the differentiation of syndromes of vascular dementia (SDSVD), ¹⁹ Activities of daily living (ADL), Hasegawa dementia scale (HDS), Functional Activities Questionnaire (FAQ), and effective rate. The effective rate of this study refers to Criteria for the Diagnosis, Syndrome Differentiation and Evaluation Standard Of Vascular Dementia, ²⁰ Diagnosis, Syndrome Differentiation and Evaluation Criteria of Senile Dementia, ²¹ and Clinical Research Guiding Principle for the New Chinese Herbal Medicine in the Treatment of Dementia.²²

Literature Exclusion Criteria

(a) Animal research; (b) SRs/MAs without quantitative synthesis, Network MAs, replication publications, case reports, conference abstracts, editorials, and narrative reviews.

Publication Screening and Data Extraction

Publication screening (SHS and ZXC) and information extraction (SHS and ZXC) were performed independently by two researchers, respectively. We first deduplicated publications in Endnote X9 bibliography management software. This was followed by an initial screening by reading the titles and abstracts of the publications. Finally, we carefully read the full text of the remaining publications to determine the final inclusion. We collected basic information on each SR/MA, including author information, year of publication, interventions, and main conclusions, using a designed data extraction form.

Quality Assessment

Two researchers (SHS and ZXC) assessed the methodological quality, reporting quality, and quality of evidence of the included SRs/MAs. The Assessment of Multiple Systematic Reviews 2 (AMSTAR-2)²³ was used to assess the methodological quality of the included SRs/MAs, and the tool consists of 16 items. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 (PRISMA 2020)²⁴ was used to assess the reporting quality of the included SRs/MAs, and it contained 27 items. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system²⁵ was used to evaluate the quality of evidence for inclusion in SRs/MAs, which was assessed from five aspects.

Meta-Analysis

We extracted data from all RCTs from the systematic evaluation included in this study. This step was independently conducted by two evaluators. Any disagreement that arose during the process were resolved by both parties through discussion. First, all RCTs were retrieved and then processed with Endnote. Afterwards, the duplicate publications were removed and all the original RCTs were downloaded for data extraction. Finally, Review Manager 5.4 was employed to conduct meta-analysis on the data and a forest map was drawn. For dichotomous outcomes, the relative risk (RR) and its 95% confidence interval (CI) were used as a summary effect measure. For continuous outcomes, the mean difference (MD) and its 95% confidence interval (CI) were used as a summary effect measure. The Cochran’s Q test and the I² statistic were applied to assess the heterogeneity among studies. And STATA 16 was used to conduct publication bias analysis on the data. Egger’s regression was used for the quantitative evaluations of publication bias.

Results

Literature Search and Selection

A total of 133 publications were searched through 7 databases and 58 duplicates were removed. After screening the literature based on titles and abstracts, the full text was assessed based on inclusion and exclusion criteria, and finally, 12 publications^26–37 were included. A flow chart of study selection was shown in Figure 1.

Figure 1 The flowchart of the screening process.

Description of Included SRs/MAs

Twelve included SRs/MAs were released between 2004 and 2022, and 8 (8/12, 66.7%)^{26,27,29,30,33–36} of these SRs/MAs were released after 2017. All the included SRs/MAs were published by Chinese, of which 2 SRs/MAs^26,27 were written in English, and the remaining 10 SRs/MAs^28–37 were written in Chinese. Each SR/MA contained several RCTs ranging from 4 to 34, and the sample size of individual studies ranged from 308 to 2672. The intervention in the control group included DT. The intervention methods of the experimental group included scalp electroacupuncture (SEA), scalp acupuncture (SA), electroacupuncture (EA), and conventional acupuncture (CA), or the above-mentioned acupuncture treatments were added on the basis of the control group. Besides that, risk of bias was assessed for eight SRs/MAs using the Cochrane tool and the Jadad Scale for the remaining four. The characteristics of the SRs/MAs included are summarized in Table 1.

Table 1 Characteristics of the Included SRs/MAs

Results of SR/MA Quality Assessment

Methodological Quality

Based on the evaluation results of AMSTAR-2, the methodological quality of all included SRs/MAs in this study was very low. The main reasons for poor methodological quality included lack of protocol registration (0/12, 0%), lack of comprehensive literature searches (4/12, 33.3%),^27,28,32,35 lack of exclusion lists (1/12, 8%),³⁵ and lack of funding source (2/12, 16.7%)^28,32 were the main downgrading factors for inclusion in SRs/MAs (Figure 2).

Figure 2 Result of the AMSTAR-2 assessment. Abbreviations: Y, Yes; PY, partial Yes; N, No.

Reporting Quality

None of the SRs/MAs fully reported studies based on the PRISMA 2020 checklist. The titles, introductions, abstracts, and discussions of the SRs/MAs included in this study were fully reported, but the completeness of reporting was otherwise unsatisfactory. The main reasons for reporting flaws included: in the methods section, lack of a complete search strategy (Item 7, 5/12, 41.7%),^{26–28,35,36} lack of sensitivity analysis (Item 13f, 4/12, 33.3%),^28,31,32,35 and lack of certainty assessment (Item 15, 2/12, 16.7%);^28,35 In the results section, lack of funding sources (Item 16b, 2/12, 16.7%)^28,35 and certainty of evidence (Item 22, 2/12, 16.7%);^28,35 in the other information section, lack of SRs/MAs protocol registration (Item 24, 0/12, 0%). More details on the quality of the report are given in Table 2.

Table 2 Results of the PRISMA Checklist

Evidence Quality

Thirty-six outcomes related to acupuncture for VaD were extracted from the included SRs/MAs. The results of the GRADE assessment showed that of the 36 outcomes, there was 1 high-quality (1/36, 2.8%) evidence, 3 moderate-quality (3/36, 8.3%) evidence, 15 low-quality (15/36, 41.7%) evidence, and 17 very low-quality (17/36, 47.2%) evidence. The main reasons for unsatisfactory quality of evidence included risk of bias (28/36, 77.8%) and publication bias (28/36, 75%). Details were listed in Table 3.

Table 3 Results of Certainty of Quality

Summary of Included Outcomes

We summarized the outcomes included in the SRs/MAs in this study as shown in Table 4.

Table 4 Summary of Outcome Measures

Effectiveness of Acupuncture on VaD Patients

Nine SRs/MAs^{26,28–35,37} reported that acupuncture could significantly improve the effective rate of VaD patients compared with the control group. Seven SRs/MAs^{26,28,30,33–36} reported the effect of acupuncture on MMSE in patients with VaD, including nine outcomes, of which seven showed that acupuncture could significantly improve MMSE in patients with VaD. Results for 9 outcomes out of 7 SRs/MAs^{26–28,30,32,34,35} indicated that acupuncture therapy could significantly improve HDS in VaD patients. Four SRs/MAs^27,33–35 reported the effect of acupuncture on ADL, and the results of two SRs/MAs indicated that acupuncture was effective in improving ADL in VaD patients. One SR/MA²⁷ result showed that acupuncture significantly reduced SDSVD and NIHSS in VaD patients.

Safety of Acupuncture on VaD Patients

There were 7 SRs/MAs^{26–28,30,32,35,36} in the results section that described no adverse events in clinical studies of acupuncture for VaD.

Summary of the Meta-Analysis

First, we searched the RCTs and removed the duplicate publications, and finally we got 110 publications. We constructed a network diagram incorporating SRs/MAs with relevant original studies (Figure 3). Then, all the original RCTs were downloaded for data extraction. In the process of information extraction, we extracted the relevant outcome indicators in GRADE evaluation for the next quantitative synthesis calculation. These outcome indicators include: effective rate, MMSE, ADL, HDS, FAQ, SDSVD, and NIHSS. In the meta-analysis, we analyzed according to acupuncture+DT vs DT and acupuncture vs DT respectively. In addition, we also conducted subgroup analysis according to the way of acupuncture. The forest plots of effective rate, MMSE, HDS, ADL, FAQ, NIHSS, and SDSVD are shown in Figures 4–7, Figure 8A–C respectively. These results also suggest that acupuncture therapy was effective rate and can be used to improve MMSE and HDS in patients with VaD. These results may also be subject to some publication bias (Table 5).

Table 5 Publication Bias of Outcome Indicators

Figure 3 Network diagram incorporating SRs/MAs with RCTs. Note: Red represents included SRs/MAs; orange represents RCTs.

Figure 4 Continued.

Figure 4 Forest plot on the effective rate of acupuncture therapy for VaD. (A) Acupuncture+DT vs DT; (B) Acupuncture vs DT.

Figure 5 Continued.

Figure 5 Forest plot on the MMSE of acupuncture therapy for VaD. (A) Acupuncture+DT vs DT; (B) Acupuncture vs DT.

Figure 6 Continued.

Figure 6 Forest plot on the HDS of acupuncture therapy for VaD. (A) Acupuncture+DT vs DT; (B) Acupuncture vs DT.

Figure 7 Continued.

Figure 7 Forest plot on the ADL of acupuncture therapy for VaD. (A) Acupuncture+DT vs DT; (B) Acupuncture vs DT.

Figure 8 Forest plot on the acupuncture vs DT of acupuncture therapy for VaD. (A) FAQ; (B) NIHSS; (C) SDSVD.

Discussion

Because there is no specific clinical treatment method, it is difficult for VaD patients to receive effective clinical treatment. Acupuncture as a potential modality for VaD management, this study comprehensively assessed the current scientific evidence for acupuncture for VaD in terms of methodological quality, reporting quality, and evidence quality.

Summary of Key Findings

This overview includes 12 SRs/MAs from 133 publications, of which 8 (8/12, 66.7%) SRs/MAs were published in the last five years (2017–2022), representing the growing interest of researchers in acupuncture interventions for VaD. From the reporting of outcome indicators, the results of 30 (30/36, 83.3%) outcome indicators indicated that VaD patients could benefit from acupuncture therapy. In addition, safety assessments indicated that acupuncture had no serious side effects. However, the methodological quality, reporting quality, and quality of evidence of the SRs/MAs included in this study were flawed, so the results of inclusion of SRs/MAs may differ from the true results and thus fail to provide reliable evidence for clinician decision-making.

Summary of Included SRs/MAs

As assessed by the methodological quality and reporting quality of the AMSTAR-2 and PRISMA 2020 scale, all SRs/MAs were of very low methodological quality and incomplete reporting. All SRs/MAs were not registered with the study protocol. A study found that the research protocol helped to increase the transparency of the methods used and improve the overall methodological quality of SRs/MAs.³⁸ In terms of literature searches, only 5 SRs/MAs reported complete database search strategies, which made publication searches less reproducible and thus less reliable. In addition, there was a lack of manual searches versus grey literature searches (4/12, 33.3%), which may increase potential publication bias. The omission of the retrieval process may directly affect the results of pooling estimated effects. Most studies (11/12, 91.7%) did not provide lists of excluded articles, which may lead authors to incorrectly exclude key articles, thereby undermining the rigour of reporting. Only 2 (2/12, 16.7%) SRs/MAs provided a full source of research funding, which may reduce the confidence in the results of RCTs, as the results of industry-funded clinical trials may benefit funders. In addition, most SRs/MAs (10/12, 83.3%) were not assessed for quality of evidence, which was the significance of this overview.

For the GRADE assessment, only one of the 36 outcome measures assessed was of high quality. The most important downgrading factors for low quality of evidence were risk of bias and publication bias. Going back to the source, most RCTs claimed to perform randomization but did not clearly state the specific method by which randomization was achieved, secondly, these clinical trials do not describe whether and how allocation concealment was performed, and finally, there was a lack of blinding of both patients and outcome assessors. The reason for the high publication bias of the included outcomes was that the number of RCTs assessed by the outcome was insufficient, so that most of the outcomes were not assessed for publication bias, leading to downgrades.

In this study we also performed a comprehensive summary of RCTs included in SRs/MAs and performed a meta-analysis of relevant outcome indicators. For the evaluation of effective rate, acupuncture showed good advantages with low heterogeneity, which shows that the evidence is more reliable. In addition to this, acupuncture therapy was also effective in the improvement of MMSE with HDS, however, there was a high level of heterogeneity, which may be explained, on the one hand, by the diversity of acupuncture point selection and, secondly, by the different ways of incorporating DT in VaD patients.

Implications for Future Research and Clinical Practice

Thromboembolism leads to decreased cerebral blood flow and chronic cerebral hypoperfusion with the development and progression of VaD, which causes cerebral hypoxia, inflammation, and oxidative stress, and ultimately leads to cognitive impairment.³⁹ Studies have shown that acupuncture can improve cognitive impairment by inhibiting inflammatory responses, anti-oxidative stress, improving vascular function and hippocampal synaptic plasticity, and promoting dopamine secretion.^40,41 In addition to this, a preclinical meta-analysis of acupuncture for VaD showed that acupuncture could enhance oxygen and glucose metabolism as well as anti-apoptotic and antioxidant properties to protect neurons during VaD.⁴²

For future SRs/MAs on this topic, it is strongly recommended to register or publish research protocols in advance on international platforms or academic journals (eg Cochrane Library, Medicine, BMJ Open, INPLASY, PROSPERO, etc.).⁴³ In the next published SRs/MAs, researchers need to supplement the grey literature search, the complete search strategy of each database, the list of excluded literature, and the list of funding support to improve the credibility and scientificity of the results.

High-quality SRs/MAs were derived from high-quality and large-sample clinical trials. Researchers should improve the top-level design of clinical research through comprehensive evaluation and detailed analysis. For acupuncture-related RCTs, the Consolidated Standard for Clinical Trial Reporting (CONSORT)⁴⁴ and the Standard for Precision Clinical Trial Intervention Reporting (STRICTA2010) should be adopted to improve the quality of evidence of RCTs and improve their guiding significance for clinical decision-making. In addition, the specificity of acupuncture treatment makes blinding of RCTs difficult. Although it is difficult to blind the clinical operator, it should be tried to blind the patient, outcome evaluators, and other caregivers, which can minimize the risk of bias. The selection of acupuncture points is diversified. With the development of evidence-based acupuncture, it is hoped that future clinical researchers will promote the standardization and accuracy of acupuncture clinical research by improving the technical operation process of acupuncture (including point selection and operation methods). Finally, in future clinical trials, when discussing acupuncture treatment of VaD, the evaluation of hemorheology, lipid metabolism and various bioactive substances related to vasoconstriction can be added on the basis of the evaluation of acupuncture efficacy and safety, so as to better explore the internal mechanism of acupuncture.

Strength and Limitations

The rapid growth in the number of SRs/MAs in acupuncture for VaD highlights the evidence-based challenges facing medical clinical decision-makers and researchers. This overview will help inform evidence-based decision-making and guide future high-quality research by assessing the methodological, reporting, and evidence quality of current high-level evidence. However, this study also inevitably has some limitations. First, the quality of the SRs/MAs included in this research was unsatisfactory, so it may not be possible to draw firm conclusions about the clinical efficacy and safety of acupuncture for VaD. Second, there may be differences in the subjective evaluations of the evaluators, leading to bias and affecting the research results.