Assessing methodological quality of systematic reviews with meta-analysis about clinical pharmacy services: A sensitivity analysis of AMSTAR-2

Abstract

Background: Systematic reviews are critical for evidence-based healthcare decisions, but their validity depends on the quality of conduct and reporting. AMSTAR-2, a widely used tool for assessing the quality of systematic reviews, identifies seven critical domains influencing review validity, although its developers recommend flexibility in prioritizing these domains. To date, no studies have analyzed the impact of this change on systematic reviews with meta-analysis (SRMAs) evaluating clinical pharmacy services. Objective: To evaluate the quality of SRMAs on clinical pharmacy services and the effect of modifying AMSTAR-2 domains criticality on quality assessment. Methods: Systematic searches (updated January 1, 2023) were conducted in PubMed, Scopus, and Web of Science to identify SRMAs reporting the effects of clinical pharmacy services. Manual reference list searches of included studies were also performed. The methodological quality of SRMAs was assessed using the AMSTAR-2 tool. Changes in the overall classification of each SRMA were analyzed by hypothetically removing the critical designation for domains in the original tool. Results: Out of 153 eligible SRMAs, 138 (90.2 %) were classified as critically low quality, 13 (8.5 %) as low quality, and 2 (1.3 %) as moderate quality. Despite slight improvement in methodological quality over time, this change was not directly linked to the creation of various reporting and conducting guidelines and registries. Our analysis showed that the hypothetical removal of the criticality of each AMSTAR-2 domain did not significantly impact the overall quality assessment. Furthermore, all critical domains in AMSTAR-2 are considered essential in the field of pharmacy practice. Conclusion: Most SRMAs on clinical pharmacy services were classified as low or critically low quality and modifying the AMSTAR-2 domain criticality did not improve these assessments. Researchers, journal editors, and peer reviewers must work to enhance SRMAs quality, which are crucial for providing robust evidence for pharmaceutical services.