Finding What Works in Health Care: Standards for Systematic Reviews
Healthcare decision makers in search of reliable information comparing health interventions increasingly turn to systematic reviews for the best summary of the evidence. Systematic reviews identify, select, assess, and synthesize the findings of similar but separate studies and can help clarify what is known and not known about the potential benefits and harms of drugs, devices, and other healthcare services. Systematic reviews can be helpful for clinicians who want to integrate research findings into their daily practices, for patients to make well-informed choices about their own care, and for professional medical societies and other organizations that develop clinical practice guidelines.
In the Medicare Improvement for Patients and Providers Act of 2008, Congress directed the Institute of Medicine (IOM) to develop standards for conducting systematic reviews and to develop standards for clinical practice guidelines, which are evidence-based recommendations for clinicians to use when treating patients. The IOM formed two distinct committees to respond to this charge, and each committee assessed the relevant evidence and considered expert guidance to develop the standards. This report,Finding What Works in Health Care: Standards for Systematic Reviews, recommends standards for systematic reviews of the comparative effectiveness of medical or surgical interventions (see a list of the standards).
Importance of Setting Standards for Systematic Reviews
The quality of systematic reviews is variable. Too often, the scientific rigor of the collected literature is not scrutinized or there are errors in data extraction and meta-analysis. Reporting biases present the greatest obstacle to collecting all relevant information on the effectiveness of an intervention. Research is important to individual decision making, whether it reveals benefits, harms, or lack of effectiveness of a health intervention. Thus, the systematic review should identify all of the studies—and all of the relevant data from the studies—that may pertain to the research question.
The task of identifying relevant research data is challenging. Although hundreds of thousands of research articles are indexed in bibliographic databases each year, a substantial proportion of effectiveness data are never published or are not easy to access. Moreover, it is well documented that published data may not represent all of the findings on an intervention’s effectiveness. Positive findings are more likely to be published than null or negative results.
In many cases, the users cannot determine the quality of a systematic review because the details of the review are so poorly documented. Additionally, many systematic reviews do not focus on questions that are important for real-world healthcare decisions, such as determining whether the benefits of taking a specific medication outweigh the risks.
Standards can improve the quality of systematic reviews, which will minimize the likelihood of clinicians coming to the wrong conclusions and ultimately making the wrong recommendation for treatment. The standards presented in this report—developed by the authoring committee—are meant to ensure objective, transparent, and scientifically valid systematic reviews. The need for establishing standards for systematic reviews was underscored in the health reform legislation The Patient Protection and Affordable Care Act of 2010, which created the nation’s first nonprofit Patient-Centered Outcomes Research Institute (PCORI).
Developing Standards for Systematic Reviews
The committee defines a “standard” as “a process, action, or procedure for performing systematic reviews that is deemed essential to producing scientifically valid, transparent, and reproducible results.”
Systematic reviews of comparative effectiveness research—a type of research that compares different treatment options for the same disease—can be narrow in scope and consist of simple comparisons, such as the effectiveness of one drug versus another. They also can address more complex questions, such as the comparative effectiveness of drugs versus surgery for a specific condition. The committee’s standards apply principally to publicly funded systematic reviews of comparative effectiveness research that focus specifically on treatments.
The evidence base for how best to conduct systematic reviews is limited, and no set of standards is generally accepted or consistently applied. For example, there is little research on how to manage bias for individuals providing input into the systematic review, or on who should screen and select studies for the review. In developing its standards, the committee relied on the current methodological evidence and guidance from respected organizations that produce systematic reviews. The committee’s standards address the entire systematic review process, from locating, screening, and selecting studies for the review, to synthesizing the findings (including meta- analysis) and assessing the overall quality of the body of evidence, to producing the final review report.
The standards are current “best practices”; they are not the last word. All of the recommended standards must be considered provisional, pending better empirical evidence about their scientific validity, feasibility, efficiency, and ultimate usefulness in healthcare decision making. The standards will be especially valuable for systematic reviews of high-stakes clinical questions with broad population impact, where the use of public funds to get the right answer justifies careful attention to the rigor of the systematic review. Individuals involved in systematic reviews should be thoughtful about all of the recommended standards and elements, using their best judgment if resources are inadequate to implement all of them, or if some seem inappropriate for the particular task or question at hand. Transparency in reporting the methods actually used and the reasoning behind the choices are among the most important of the standards recommended by the committee.
Improving the Quality of Systematic Reviews
The committee proposes a framework for improving the quality of the research that supports systematic reviews, including strategies for involving the right people, methods for conducting the systematic review, methods for synthesizing and evaluating evidence, and methods for communicating and using the results. Successful execution and effective use of a systematic review requires improving the science supporting the steps in the systematic review process.
In addition, the committee finds that the environment surrounding the development of systematic reviews lacks adequate funding and coordination—both of which are needed to conduct high-quality systematic reviews. Many organizations conduct systematic reviews, but typically they do not work together. The committee emphasizes the need for greater collaboration among stakeholder groups, including PCORI, government agencies, medical professional societies, researchers, and patient interest groups. Together, these groups have the potential to improve the rigor and transparency of systematic reviews, encourage standardization of methods and processes, set priorities for selection of clinical topics of interest to clinicians and patients, reduce unintentional duplication of efforts, and more effectively manage conflicts of interest.
Recommendations
The committee recommends that PCORI provide oversight and encourage coordination among Department of Health and Human Services (HHS) agencies to improve the research base and support the environment for systematic reviews. Improved coordination should include:
- Developing training programs for researchers, users, consumers, and other stakeholders to encourage more effective and inclusive contributions to systematic reviews of comparative effectiveness research;
- Systematically supporting research that advances the methods for designing and conducting systematic reviews of comparative effectiveness research;
- Supporting research to improve the communication and use of systematic reviews of comparative effectiveness research in clinical decision making;
- Developing effective coordination and collaboration between U.S. and international partners;
- Developing a process to ensure that standards for systematic reviews of comparative effectiveness research are regularly updated to reflect current best practice; and
- Using systematic reviews to inform priorities and methods for primary comparative effectiveness research.
Conclusion
Systematic reviews should be used to inform healthcare decision makers about what is known and not known about the effectiveness of health interventions. Patients expect that their doctors and other healthcare providers know what type of treatment to recommend. Yet the reality is that the evidence that informs current healthcare decisions often is incomplete and may be biased, and there are no standards in place to ensure that systematic reviews of the evidence are objective, transparent, and scientifically valid. Better-quality systematic reviews have the potential to improve the decisions made by clinicians, to better inform patient choice, and to provide a more trustworthy basis for decisions by payers and policy makers.
No hay comentarios:
Publicar un comentario