Health Economics Information Resources: A Self-Study Course
Module 4: An Introduction to the Principles of Critical Appraisal of Health Economic Evaluation Studies
Sample critical appraisal exercise: An economic evaluation of thrombolysis in the community, continued...
6. Were costs and consequences valued credibly?
6.1. Were the sources of all values clearly identified? (Possible sources include market values, patient or client preferences and views, policy-makers’ views and health professionals’ judgments)
Market values used for drug and capital costs and year stated as 1996 (in Table 1). Labor costs, in this case GP time, based on recommended income scales.
No year or reference is given.
6.2. Were market values employed for changes involving resources gained or depleted?
Market values for capital costs were adjusted using a discount rate of 6%. Adjustment for inflation for costs not required here as they all occur at the same time.
6.3. Where market values were absent (e.g., volunteer labor), or market values did not reflect actual values (such as clinic space donated at a reduced rate), were adjustments made to approximate market values?
6.4. Was the valuation of consequences appropriate for the question posed (i.e., has the appropriate type or types of analysis – cost-effectiveness, cost-benefit, cost-utility – been selected)?
Consequences were measured as life years saved; i.e., outcome measured as survival at 4 years - this is appropriate for an incremental cost-effectiveness analysis.
Consequences measured as quantity of life but consequences were not weighted by quality of life and therefore not best suited to evaluation of an allocative question.
7. Were costs and consequences adjusted for differential timing?
7.1. Were costs and consequences that occur in the future ‘discounted’ to their present values?
Capital costs discounted at 6%. Discounting not applicable to the other costs included because they occur at the same time.
7.2. Was there any justification given for the discount rate used?
No justification provided for the discount rate used. 6% was the UK recommended rate.
8. Was an incremental analysis of costs and consequences of alternatives performed?
8.1. Were the additional (incremental) costs generated by one alternative over another compared to the additional effects, benefits, or utilities generated?
Yes. This gives a meaningful comparison in terms of the additional cost and effect of one intervention over another.
A value judgment will be required to assess whether the extra unit of outcome is worthwhile. This provides some information in relation to whether the extra benefit is worth the extra resources.
9. Was allowance made for uncertainty in the estimates of costs and consequences?
9.1 If data on costs and consequences were stochastic (randomly determined sequence of observations), were appropriate statistical analyses performed?
The only stochastic tests were performed on consequences; i.e., probability of survival at 4 years - confidence intervals. In this case the the additional probability of survival at 4 years is 11% (95% CI* 1% to 22%).
This suggests that the true value of additional probability of survival at 4 years will be found to be between 1% and 22%, 95 out of a hundred times.
* CI = Confidence Interval
9.2. If a sensitivity analysis was employed, was justification provided for the range of values (or for key study parameters)?
The study explored the impact of assumptions with respect to resource use. Low and high cost estimates provided around assumptions of time cost for GP attendance and capital expenditure.
Lower estimate based on assumption that no capital expenditure was necessary and that GPs already attended patients with suspected AMI.
Higher estimate based on assumption that an ECG and defibrillator would need to be purchased and that the GP did not previously attend patients with suspected AMI.
Capital costs were the only variable tested. Other variables which could have been tested include thrombolytic drug costs and GP time.
9.3. Were the study results sensitive to changes in the values (within the assumed range for sensitivity analysis, or within the confidence interval around the ratio of costs to consequences)?
Yes, the results were sensitive to the change in values.
Extreme values of marginal cost of life saved at 4 years, i.e., £1990 - £88100, were not particularly precise. This is because the trial was originally designed to detect changes in survival, not determine cost-effectiveness.
10. Did the presentation and discussion of study results include all issues of concern to users?
10.1. Were the conclusions of the analysis based on some overall index or ratio of costs to consequences (e.g., cost-effectiveness ratio)? If so, was the index interpreted intelligently or in a mechanistic fashion?
Conclusions of the analysis based on an incremental cost-effectiveness ratio.
Indicates how much was paid for each additional life saved (at 4 years) by use of community thrombolysis.
10.2. Were the results compared with those of others who have investigated the same question? If so, were allowances made for potential differences in study methodology?
No, results not compared with those of others who have investigated the same question, if indeed anyone has. This would require an individual to identify and check other studies.
10.3. Did the study discuss the generalizability of the results to other settings and patient/client groups?
This issue was not explored.
10.4. Did the study allude to, or take account of, other important factors in the choice or decision under consideration (e.g., distribution of costs and consequences, or relevant ethical issues)?
If community thrombolysis is adopted, within a fixed budget, less people will be treated (because it is more expensive), but those who are treated will be better off (in terms of survival). If the aim is to address technical efficiency, x no of people will not receive thrombolysis (either in hospital or in the community) and their ‘fate’ will be unknown.
Therefore the results of the study are not sufficient to address technical efficiency.
This raises issues of equity.
10.5. Did the study discuss issues of implementation, such as the feasibility of adopting the ‘preferred’ program given existing financial or other constraints, and whether any freed resources could be redeployed to other worthwhile programs?
Yes, the study discussed some issues of implementation mainly transfer of workload from secondary to primary care and the potential need for incentives (financial or non-financial) for GPs to take on this work.
More generally, it states the need for extra resources in order to implement such a service. Given that the study is from the health care payer perspective, the relevant decision is whether the extra resources required could be released from other programs without losing benefits that are greater than those provided by community thrombolysis.