National Information Center on Health Services Research and Health Care Technology (NICHSR)
Serving the Information Needs of the Health Services Research Community
HSR General Resources
- Data, Tools and Statistics
- HSR Social Media Resources
- Education and Training
- Grants, Funding and Fellowships
- Guidelines, Journals Other Publications
- Key Organizations
- Meetings and Conferences
- State and State/Local Resources
- Aging Population Issues
- Community Benefit / Community Health Needs Assessment
- Comparative Effectiveness Research (CER)
- Child Health Services Research
- Evaluation Resources for Assessing HIT Systems and HIT Implementation, Adoption and Use
- Dissemination and Implementation Science
- Evidence-Based Practice and Health Technology Assessment
- Health Care Reform, Health Economics, and Health Policy
- Health Disparities
- Health Informatics
- Health Literacy and Cultural Competence
- Privacy/Security and Research with Electronic Health Records
- Public Health Services and Systems Research
- Rural Health
Evaluation Resources for Assessing HIT Systems and HIT Implementation, Adoption and Use
With the large investments required for health IT projects, stakeholders are increasingly demanding to know what the actual value of these projects has been, or will be. Evaluations allow us to determine whether or not what one has set out to accomplish has been accomplished, and to help us to understand what has worked in a given project and what has not. Lessons that emerge from evaluations help to guide both your team, as well as others, in their approach to projects in the future. Evaluations must therefore be an integral part of any health IT project. (AHRQ Health IT Evaluation Toolkit)
The diversity and complexity of health IT projects poses challenges for those seeking to evaluate particular projects and those interested in assessing broader population impact. While support for such evaluations can be found in ongoing health service research projects (explore the database Health Services Research Projects in Progress, as well in the proliferation of Toolkits from diverse public and non-profit sources (see below), there is also an increasing literature assessing evaluation techniques and approaches appropriate for health IT interventions, particularly those related to implementation of the requirements for "Meaningful Use." This literature can be found using PubMed search strategies, such as those at http://www.nlm.nih.gov/healthit/meaningful_use.html.
- Acceptance and Approval of Non-Governmental Developed Test Procedures, Test Tools, and Test Data for Use Under the ONC Health IT Certification Program--15-JUN-2015 - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - This document from the Federal Register was reissued on June 9, 2015 and reiterates ONC's policy that permits any person or entity to submit test procedures, test tools, and test data for approval and use under the ONC Health IT Certification Program.
- EHR Toolkit - Behavioral Health Providers - (HealthInfoNet USA) - Implementation tools have been adapted for use by behavioral health providers.
- Evaluating, Optimizing, and Sustaining - (Health Resources and Services Administration (HRSA)) - This resource, from the Health IT Implementation Toolbox, is designed for health centers and safety net and ambulatory care providers who are seeking to implement health IT.
- Guide to Reducing Unintended Consequences of Electronic Health Records - (Agency for Healthcare Research and Quality (AHRQ)) - This resource was developed for all types of health care organizations to help minimize avoid problems during EHR implementation.
- Health Information Technology (HIT) Resources for the Delta Rural Hospital Performance Improvement (RHPI) Program - (Upper Midwest Rural Health Policy Center USA) - This toolkit assists rural health providers seeking to implement HIT to improve the overall effectiveness of their institutions.
- Health Information Technology Technology Toolkit for Critical Access and Small Hospitals - (StratisHealth USA) - This toolkit was designed to assist small hospitals for different points in HIT adoption.
- Health IT Implementation Toolkit - (Health Resources and Services Administration (HRSA)) - This toolkit contains a compilation of planning, implementation and evaluation resources.
- Health IT Literacy Guide - (Agency for Healthcare Research and Quality (AHRQ)) - This guide provides strategies, and other resources for the development of technical standards and aspects of accessible health IT design,
- Health IT Patient Satisfaction Survey - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - This survey tool is intended to aid providers and health IT implementers assess the impact of health information technology adoption to inform quality improvement initiatives. The results will provide an understanding of how patients are responding to EHR use in the office.
- Health IT Survey Compendium - (Agency for Healthcare Research and Quality (AHRQ)) - This database is a resource of health IT surveys that may be used in the evaluation of health IT projects.
- The HIMSS Health IT Value Suite - (Healthcare Information Management Systems Society (HIMSS) USA) - This is a growing database of use studies, related to health-IT.
- How to Implement EHRs - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - This is a step-by-step overview of the tasks necessary to implement an EHR system.
- Implementation Resources - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - A series of tools, webinars, reports, and training modules designed to assist in the EHR process.
- ONC HIT Curriculum Overview - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - Registration is required to utilize this free curriculum. Included in the course are Working with Health IT Systems, Configuring Electronic Health Records, and Planning, Management and Leadership for Health IT.
- Post EHR Implementation Evaluation - (HealthInfoNet USA) - This worksheet is designed to assist in the post-implementation evaluation process.
- Resources for Critical Access Hospitals and Small Rural Hospitals - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - Designed for rural hospitals, this toolkit discusses the benefits of health IT and helps hospitals to take the first step toward implementation and attaining meaningful use.
- Rural Health IT Adoption Toolbox - (Health Resources and Services Administration (HRSA)) - Developed for rural health providers, this toolbox is designed to improve effectiveness of the HIT implementation process.
- Workflow Assessment for Health IT Toolkit - (Agency for Healthcare Research and Quality (AHRQ)) - Ambulatory care centers are the focus of the workflow assessment toolkit.
- Demonstrating Impact of a Model Pediatric EHR Format - (Agency for Healthcare Research and Quality (AHRQ)) - This model EHR record format for children is the result of a grant to 10 states, and several states will be evaluating the project.
- Evaluation of the Flexibility Critical Access Hospital Health: Information Technology Network Implementation Program - (Health Resources and Services Administration (HRSA)) - From December 2010, this report outlines the work of 16 rural institutions, which developed and implemented HIT pilot networks.
- Executive Summary: Sustainability, Partnership, and Teamwork in Health IT Implementation: Findings From the Transforming Healthcare Quality Through IT Grants (2012) - (Mathematica Policy Research, Inc.) - This report presents an overview of the Transforming Healthcare Quality Through IT grantees' experience.
- GEP-HI Good Evaluation Practice Guidelines for Health Informatics - This article from the International Journal of Medical Informatics outlines the development of a practice guideline to plan and perform evaluation studies in health informatics.
- Health Information Technology and Health Information Exchange Implementation in Rural and Underserved Areas: Findings from the AHRQ Health IT Portfolio (2010) - (Healthcare Information Management Systems Society (HIMSS) USA) - This report focuses on the challenges facing rural and underserved communities in integrating health IT into their health care delivery systems
- Health Information Technology Approaches in QUERI Implementation Research: Case Study Evaluation - (Veterans Health Administration US) - From August 15, 2011 this follow-up study uses a case study approach to examine HIT strategies in use, barriers to, facilitators to, and requirements for HIT use
- Health Information Technology Publications - (National Association of State Chief Information Officers (NASCIO)) - Reports on state health information technology initiatives.
- Health Information Technology: Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology, 2014 Edition; Revisions to the Permanent Certification Program for Health Information Technology - (Department of Health and Human Services (HHS) U.S.) - The Final Rule, dated September 4, 2012, contains standards for Privacy and Security Certification and on the technical capabilities and implementation standards for the EHR.
- Health IT Evaluation Measures: Quick Reference Guides - (Agency for Healthcare Research and Quality (AHRQ)) - Quick Reference Guides are designed provide details about individual measures that can be incorporated into a health IT evaluation plan.
- HealthIT Dashboard - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - The Dashboard provides quick access to federal policies, programs, and research activities related to health IT.
- How do I Conduct a Post-Implementation Evaluation? - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - This outline briefly overviews the steps needed in conducting an post-implementation evaluation.
- A Human Factors Guide to Enhance EHR Usability of Critical User Interactions when Supporting Pediatric Patient Care Implementation - (National Institute of Standards and Technology (NIST) US) - Recommendations to enhance EHR usability when supporting pediatric patient care are presented, as well as identifying areas for EHR innovation.
- Lessons from the Literature on Electronic Health Record Implementation - (Urban Institute (UI)) - The results of a study that looks at published and grey literature to determine the best practices in implementing health IT.
- Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices - (Agency for Healthcare Research and Quality (AHRQ)) - The information in this report is intended to help health care decisionmakers make well-informed decisions and thereby improve the quality of health care services.
- STARE-HI - Statement of Reporting of Evaluation Studies in Health Informatics - This is a guideline for the writing and assessing evaluation studies on HIT. It has been approved by International Medical Informatics Association and the European Federation for Medical Informatics.
- STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics: Explanation and Elaboration - This 2013 article from Applied Clinical Informatics elaborates on the 2009 STARE-HI study. Its aim is to guide those using evaluation studies in health informatics by providing explanatory examples of reporting.
- The State HIE Program Four Years Later: Key Findings on Grantees' Experiences from a Six-State Review (2014) - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - This Case Study Report presents the results of a NORC series of case studies, which identified key enablers, challenges, and lessons learned from the State Health Information Exchange Cooperative Agreement Program. The report presents the results of a survey and site visit of the implementation experiences in six states (Iowa, Mississippi, New Hampshire, Utah, Vermont, and Wyoming).
- Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records (2012) - (National Institute of Standards and Technology (NIST) US) - This is a validation study procedure which guides the reader to ensure that the user interface is free from critical usability issues and supports error-free user interaction with EHR.
- Anatomy of Implementation: The Structural Framework for Meaningful Use of Electronic Health Records (2012) - (National Health Care for the Homeless Council USA) - This document presents a framework on the successful implementation of an EHR system.
- Community Health Centers and Electronic Health Records: Issues , Challenges, and Opportunities - (Kaiser Permanente US) - This handbook contains a framework for EHR adoption.
- Electronic Health Record Usability: Evaluation and Use Case Framework (2009) - (Agency for Healthcare Research and Quality (AHRQ)) - This study establishes a foundation for user interface design considerations and proposes an action agenda for the application of information design principles to the use of HIT in primary care settings.
- Meaningful Use and Critical Access Hospitals: A Primer on HIT Adoption in the Rural Health Care Setting - (Health Resources and Services Administration (HRSA)) - Issued in December 2010, this is a implementation process framework for successful adoption and use.
- New Funding Announcements - (Office of the National Coordinator for Health Information Technology, HHS (ONC))
- New Funding Announcements - ONC - (Office of the National Coordinator for Health Information Technology, HHS (ONC))
- EHR Usability & Patient Safety Roundtable: "Supporting Patient Safety through EHR Design" - (National Institute of Standards and Technology (NIST) US) - Slides from the presentations made at the meeting on April 19, 2013 are available.
- Methodological Considerations When Evaluating the Implementation of Large‐Scale Electronic Health Record Systems - (AcademyHealth) - A slide presentation from the 24 June 2012 AcademyHealth Annual Research Meeting, this document looks at the challenges of EHR evaluations.
- Usability in Health IT: Technical Strategy, Research, and Implementation - (National Institute of Standards and Technology (NIST) US) - A summary of the roundtable discussion on improving the usability and accessibility of EHRs, held on July 13, 2010 is available.
- Access to Capital - Rural Hospital - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - To prepare rural hospital for the questions they will face when applying for a loan, this webinar explains the evaluation process commercial banks use to make a decision on a loan.
- HRSA Webinars - (Health Resources and Services Administration (HRSA)) - Health IT and Quality webinars are technical assistance for HRSA grantees and other safety net providers.
- AHRQ National Resource Center for Health Information Technology - (Agency for Healthcare Research and Quality (AHRQ)) - AHRQ initiative containing information on programs that promote the use of health information technology. Central repository for research findings from projects developing, testing, and using health IT applications.
- Health Resources and Services Administration (HRSA) - HRSA works to improve and extend life for people living with HIV/AIDS, provide primary health care to medically underserved people, serve women and children through state programs, and train a health workforce that is both diverse and motivated to work in underserved communities.
- Healthcare Information and Management Systems Society (HIMSS) - (Healthcare Information and Management Systems Society) - The healthcare industry's membership organization exclusively focused on providing leadership for the optimal use of healthcare information technology (IT) and management systems for the betterment of healthcare.
- HealthIT.gov - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - Resource to support the adoption of health information technology and the promotion of nationwide health information exchange to improve health care.
- IHE USA - IHE USA is a part of the international group, and focuses on fostering consistent information standards.
- National Institute of Standards and Technology - NIST is the U.S. physical science laboratory, and, one of their primary concerns is information technology.
- Regional Extension Centers (RECs) - (Office of the National Coordinator for Health Information Technology, HHS (ONC)) - Information is provided on the ONC's Regional Extension Centers, which serve as a support and resource centers to assist providers in EHR implementation and HealthIT needs