U.S. National Library of Medicine

Introduction to Health Services Research : A Self-Study Course

Module 5: Quality Filtering and Evidence-Based Medicine and Health (page 9 of 15)

Analysis (inference)

Once the measurements and data are collected, it's time to analyze the results! Researchers perform analysis to identify three major characteristics.

Strength of the Association

How strong was the association between variables? For example, how strongly was length of stay (LOS) related to the likelihood of readmission to a hospital? How strongly is smoking associated with lung cancer?

Relative risk compares the risk or probability of one group having a particular outcome with the risk of another group having that outcome given the presence or absence of an exposure or experience.

These estimations of strength can be expressed in different ways. A common expression is the relative risk. Relative risk compares the risk or probability of one group having a particular outcome with the risk of another group having that outcome given the presence or absence of an exposure or experience.

In the first example, a researcher would compare a group which has a two day LOS with a group which has a six day LOS for the same condition. She would calculate the number of people in each group who required readmission. If 10 out of 20 people in Group 1 required readmission, their risk (or probability) of readmission is 10/20 or 0.50 (50 percent). If 2 out of 20 people in Group 2 required readmission, their risk of readmission is 2/20 or 0.10 (10 percent). The relative risk comparing each group's risk is .50/.10 or 5. Group 1 has five times the risk or likelihood of needing readmission with a stay of two days compared to Group 2 with a length of stay of six days.

Relative risks of 5 are considered quite substantial. Many risks in epidemiological studies are usually much smaller (around 2), leading to much controversy in interpreting the risk as meaningful.

Statistical Significance

What is the likelihood of getting the results from our sample if there was no relationship between variables in the larger population from which the sample came? For example, our group of people showed that short lengths of stay were associated with a greater probability of readmission. Is this result likely to be caused by chance or by something real?

Statistical testing is used to calculate the probability of getting the results from our study sample if there is no association between lengths of stay and the rate of readmission (the null hypothesis of no association between the variables). If this probability is so small, we decide that the null hypothesis is wrong -- that there is a relationship between our variables.

Statistical significance or inference testing deals with the characteristics of the larger population. Researchers infer what is happening in the larger group by examining a smaller sample. The measure of significiance is the p value, often expressed as p < .05 or p = .01.

A p value less than .05 means that our results have less than a 5 percent chance of occurring if there is no relationship in the larger population. A p value of .01 means that our results have a 1 percent probability of occurring if there is no relationship in the larger population. At these small probabilities, most researchers conclude that this chance is too small. Therefore, they accept that there is a relationship in the larger population.

Notice that evidence of a relationship or association is NOT the same as evidence of cause and effect. More evidence is needed to conclude that two things are causally related. Having statistical significance does not confer clinical significance to a study result. The relationship between 1500 calories a day and 1550 calories a day may be statistically significant in a study of 1000 people trying to lose weight. It probably does not make a difference in real life, and thus would not be considered "clinically significant."

If the probability of getting a relative risk of 5 in the above example had a p value of .05, the researcher could aptly conclude that there is a relationship between LOS and hospital readmission in the larger group. However, she could not say that LOS caused this increase in readmissions until she checked other factors, such as the similarity between the groups she studied.

Adjustment

Were the groups in the study different in any way that could affect the results? For example, if the patients in Group 1 had more complications or co-existing conditions than the patients in Group 2, they may have needed readmission because of these conditions and not because of their short length of stay.

Factors that create group differences and affect the study's results are called confounders. Researchers have several different types of tools to analyze confounders and determine if they biased the study. These techniques include stratification and regression analysis.

In an article on the effects of pollution on heart failure, the authors carefully describe the possible confounders that they identified:

Time variant factors with the potential to confound the observed association include temperature, seasonal weather patterns, and outdoor activity patterns. ...Exposure to other pollutants may also confound the association between a single pollutant and congestive heart failure. (Morris, 1995).

Understanding the Analysis Section in Research Articles

The Analysis section of research papers is often difficult to understand without a statistical background. Therefore, more training or consultation with statisticians will increase comprehension and critical appraisal of research studies.

The Analysis section of research papers is often difficult to understand without a statistical background. Many statistical tests can be performed only under specific circumstances, and these conditions are not known to casual readers. Therefore, more training or consultation with statisticians will increase comprehension and critical appraisal of research studies.

Discussion Questions

1. In doing critical appraisals have you found that the Analysis section of research papers is difficult to understand? Give an example of why this section may be difficult for those not trained in statistical methods to understand.
2. How important is thoroughly understanding the concept of "null hypothesis" to your critical appraisal skills? Where would you go to get additional information on this concept if you forgot what it meant?
3. Is it helpful to have a glossary of statistical terms available as you read over research papers? What resource do you use?