Skip Navigation Bar

National Information Center on Health Services Research and Health Care Technology (NICHSR)

spacer
spacer
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
blue arrow
spacer
blue arrow facing left Previous   Next blue arrow

Introduction to Health Services Research : A Self-Study Course

Module 5: Quality Filtering and Evidence-Based Medicine and Health (page 8 of 15)
Introduction | Sampling | Assignment | Assessment | Analysis | Interpretation | Extrapolation

Assessment (measurement)

What if a researcher assessed a person's blood pressure with an x-ray or asked people with Alzheimer's to remember their previous diet? No one could trust a study that had such inaccurate measurements. Accurate and complete measurements are essential; unfortunately, there are are many ways to mismeasure variables when people are the object of investigation!

Recall Bias

For example, interviewers have questioned farmers with and without cancer about their exposure to pesticides. Farmers with cancer may be more likely to remember their pesticide use than the healthy farmers since people who have suffered a traumatic event try harder to find a reason for it. This recall bias distorts the assessment of exposure since both groups differ in their ability to remember past events.

Recall bias is common in studies that compare remembered experiences of ill and healthy people.

Measurement Errors

Other errors in measurement occur when the instruments themselves are inaccurate, when people in the study drop out and cannot be followed (lost to follow-up), when study participants change their behavior because they are under observation, and when the investigators question one group more thoroughly or more frequently than the other. Masking (also called "blinding") helps to minimize these errors in clinical trials. With this technique, the participants, researchers, and/or data managers are unaware of the assignment of the participants.

Data Issues

In health services research, researchers often rely on data collected by government agencies or owned by proprietary enterprises. They may not have the full picture to assess unemployment rates when the definition of unemployment changes during the study period. Or, it may be difficult to compare cost savings between a managed care company and a Medicare provider when one dataset is private and the other is available publicly.

Comparing Outcomes

It is also difficult to compare outcomes of specific procedures across hospitals because the patients may differ so much (case-mix). In one study on the access and quality of coronary artery bypass surgery (CABS) in the United States and Canada, the authors acknowledge the difficulties in measuring people accurately:

Differences in coding conventions and other aspects of data collection and recording among the different state and provincial discharge abstract systems, as well as limited clinical detail, present problems for measuring differences in case mix when analyzing mortality across jurisdictions. For example, CABS performed as an emergency procedure for complications of percutaneous transluminal coronary angioplasty (PTCA) carries a high operative mortality. However, our comparison of a sample of Canadian discharge abstracts with corresponding physician billing data revealed that PTCA was markedly undercoded...

Because of these method limitations, ... we restricted our overall case-mix adjustment to controlling for age and sex only. (Grumbach, 1995).

Discussion Questions

  1. When reading the above study on study on the access and quality of coronary artery bypass surgery (CABS) some questions might have occurred to you. Note how people or factors are measured. Does it make sense?
  2. When evaluating a study, do the researchers account for everyone in the study or do they ignore those who drop out? Why are dropouts important in a study?
  3. Are abstract concepts (like "quality of life") defined and measured concretely? Why should these abstract concepts be measured concretely?
  4. Several ways measurement errors may occur were listed above and repeated here:
    • the instruments themselves are inaccurate,
    • people in the study drop out and cannot be followed (lost to follow-up),
    • study participants change their behavior because they are under observation, and
    • the investigators question one group more thoroughly or more frequently than the other
  5. What other measurement errors have you read about elsewhere? Where would you go to find out about other measurement errors?
blue arrow facing left Previous Knowledge Check
Q11 | Q12
Next blue arrow
spacer
spacer