Health Indicators Overview: Questions and Answers

1. Do you know of any good international surveys/survey data?

There are many surveys associated with international health and USAID projects, e.g., the Demographics and Health Program. These assess overall social demographics, reproductive health, infectious disease risks, nutrition, etc. all over the world. The Centers for Disease Control and Prevention (CDC) provides technical assistance in the collection and analysis of international survey data, so that may be a good resource. The Organization for Economic Cooperation and Development (OECD) offers a wealth of comparative data by nation. The World Health Organization is also a good source.

2. I've heard criticisms of the Dartmouth methodology and wondered if the methodology/data is valid?

The Dartmouth Atlas and related web tools look at variation in the use of medical resources and outcomes as well as comparative measures of effectiveness and quality of healthcare. The producers of the Atlas are highly qualified researchers with a strong track record of using peer reviewed research processes. The Atlas uses Medicare data that is largely based upon claims and is therefore subject to limitations of those data. However, the Dartmouth research is also based on years of scientific study and research methodology. The Atlas calls into question unwarranted variation in health care spending and quality of health care and this is going to be controversial. It challenges a number of practices that result in higher spending and that don't necessarily improve patient outcomes but in fact may worsen patient outcomes.

There is a broader issue with this question as it pertains to indicators and that is the importance of understanding the strengths and limitations of any data source, including claims data. For example, surveys can have differential biases associated with self-reporting on various questions; disease surveillance systems and resulting incidence rates can be biased because of incomplete or differential physician reporting by condition.

3. How do you know or how can you tell if an index or indicator set is a good one?

Basic evaluative criteria requires understanding the expertise of the compiling organizations, the quality/completeness of the data, and how broadly the data are used. One should always look for and review published methodology. Once these basic requirements have been satisfied, the construction of a set of indicators is really based on the purpose of the indicators - how are they to be used, what do the compilers think are the most compelling indicators - analogous to the "proxy power" concept from Mark Friedman. That is, does the indicator say something meaningful, of central importance, about health, and is a change in that indicator associated with better (or worse) health? There are many indicators that would meet this test and then the work becomes making difficult choices. Any room full of people, especially data people, will never agree 100% on such choices.

4. What is the difference between an indicator and a measure?

An indicator [is a type of measure that] provides evidence that a certain condition exists or that tells how well certain results or objectives have or have not been achieved (Brizius & Campbell, p.A-15). Indicators enable decision-makers to assess progress towards the achievement of intended outputs, outcomes, goals, and objectives. Horsch, K. Indicators: Definition and Use in a Results-Based Accountability System. Cambridge, MA: Harvard Family Research Project.I

5. Are there any proxies for social connectedness? What types of measures (or validated instruments) are available for social connectedness and social support?

This is difficult to summarize succinctly as there are many types of measures and this is an evolving field. Some validated questions have been used to assess the number of close friends or family members that would help with emotional problems or feelings when needed. Some are based on the frequency - or how often people feel that they have someone to turn to for help with an emotional problem or feelings. Other questions are in the context of the family and assess social support and connectedness in terms of the role as parents (number of people, or ease or difficulty of finding people, whom you can turn to for advice about your child), still others involve social connectedness in a broad community context (e.g., community belonging, interest in knowing neighbors, number of neighbors one can ask for help). The Behavioral Risk Factor Surveillance System (BRFSS) measures and some of the more prominent child/family surveys in the U.S. and in Canada are good places to search for validated items.

6. I understand local states can add questions to the BRFSS. What is the process for having a question included on the survey?

The Centers for Disease Control and Prevention's Behavioral Risk Factor Surveillance System Web site provides information on how to add questions to the BRFSS.

7. You mentioned a review of US indicators called the SUSA Review, is this the correct name for it?

The State of the USA is a set of broad national indicators and data, e.g., education indicators, economic indicators, health indicators, etc., currently under development. The health indicators for SUSA were carefully selected by the National Academy of Medicine's Committee on State of the USA Health Indicators. The Committee's report describes the 20 health indicators used in SUSA and the selection process. Cheryl Wold, the presenter for this webinar, compiled a report for the NAM Committee entitled Health Indicators: A Review of Reports Currently in Use (July 2008).

8. When foundations are involved in the indicator data reporting, is there concern on whether ideologic leanings will influence the results?

Any presentation of information can be influenced by choices which are in turn influenced by one's ideologies. That said, information shown on conceptual frameworks helps to convey that the field of indicators is based upon frameworks that are linked to factors that have been shown by research to impact health. Most health indicator efforts are rooted in the notion that health disparities should improve and the belief that tracking progress using high-quality data will help to urge improvements. It's true that foundations come from a broad spectrum of ideological perspectives. Those that are heavily involved in research and data dissemination are interested in evidence and results, which are increasingly broadly embraced and not the purview of any particular political belief or ideology. In the end, statistics can be manipulated in their presentation and interpretation, so it's important to be good consumers of that information.

9. Are indicator data typically updated annually? When is 2010 data likely to be available?

Most national reports are updated annually. It is not uncommon for local reports to be updated every two years. The availability of data varies highly. The vast majority of data have significant "lag times" because completing data collection, entry, validation, etc., takes time. For example, vital records generally have at least a two year lag time; for surveys, it is closer to a year.