National Information Center on Health Services Research and Health Care Technology (NICHSR)
Health Economics Information Resources: A Self-Study Course
Module 4: An Introduction to the Principles of Critical Appraisal of Health Economic Evaluation Studies
Sample critical appraisal exercise: An economic evaluation of thrombolysis in the community
1. Was a well-defined question posed in answerable form?
1.1. Did the study examine both costs and effects of the service(s) or program(s)?
1.2. Did the study involve a comparison of alternatives?
1.3. Was a viewpoint for the analysis stated and was the study placed in any particular decision-making context?
2. Was a comprehensive description of the competing alternatives given (i.e., can you tell who did what to whom, where, and how often)?
2.1. Were there any important alternatives omitted?
2.2. Was (should) a do-nothing* alternative be considered?
3. Was the effectiveness of the program or services established?
3.1a. Was this done through a randomized controlled trial (RCT)?
3.1b. If so, did the trial protocol reflect what would have happened in regular practice?
3.2. Was the effectiveness established through an overview of clinical studies?
3.3. Were observational data or assumptions used to establish effectiveness? If so, what are the potential biases in results?
4. Were all the important and relevant costs and consequences measured accurately in appropriate physical units (e.g., hours of nursing time, number of physician visits, lost work-days, gained life years)?
4.1. Was the range wide enough for the research question at hand?
4.2. Did it cover all relevant viewpoints? (Possible viewpoints include the community or social viewpoint, and those of patients and third-party payers. Other viewpoints may also be relevant depending upon the particular analysis.)
4.3. Were the capital costs, as well as operating costs, included?
5. Were costs and consequences measured accurately in appropriate physical units (e.g., hours of nursing time, number of physician visits, lost work-days, gained life years)?
5.1. Were any of the identified items omitted from measurement? If so, does this mean that they carried no weight in the subsequent analysis?
5.2. Were there any special circumstances (e.g., joint use of resources) that made measurement difficult? Were these circumstances handled appropriately?