CAHO’s Evaluation of ARTIC Program

Phase I Evaluation of the ARTIC Program 

CAHO recently conducted an Evaluation of the ARTIC Program, to assess if the program is meeting its objectives. This is meant to shape the future of the ARTIC Program and ensure that it is providing the right mix of supports to effectively facilitate the rapid movement of evidence into practice.

The key findings of the Report, released on November 15, 2013, are:

  • ARTIC’s design and structure support, enhance and accelerate the implementation of research evidence into Ontario’s health care system, the Program’s primary objective.
  • The ARTIC Program is well positioned to make a significant impact on quality improvements in Ontario’s health care system going forward.
  • The guidance and management from the centralized program office have been invaluable to the successful implementation of ARTIC Projects and have increased standardization and quality across projects.

The evaluation also identified areas for improvement. According to the evaluation, the ARTIC program is capable of spread and scale beyond CAHO hospitals. This includes the documentation and sharing of guidelines with a broader community of stakeholders and organizations. Other recommendations include the adoption of regular feedback mechanisms for all projects, as well as the definition and adoption of standardized metrics for measuring value, impact and sustainability across all current and future.

CAHO engaged Cathexis Consulting to conduct the evaluation and produce a report. The full report, by Cathexis Consulting Inc., is attached for your reference: CAHO ARTIC Phase 1 Evaluation Final Report.

Phase II Evaluation of the ARTIC Program 

CAHO recently conducted the Phase II evaluation to assess the impact, value and sustainability of the ARTIC Program. The Phase II Evaluation of the ARTIC Program had six key findings:

1. The ARTIC program made implementation of evidence-based interventions more likely, faster and more consistent with the research evidence.

2. The ARTIC program supported high levels of sustainment.

  • The first four ARTIC projects were fully sustained in 76% of the sites. This is an impressive sustainment rate, when compared with other relevant Ontario-based multi-site studies that cite sustainment rates of 50-60%.

3. Directly through ARTIC, the 6 projects were spread to 79 sites and beyond.

  • An unexpected, but very positive, outcome of ARTIC was extensive spread of its projects’ interventions to more than 200 additional hospital sites. The ARTIC program acted as a spread agent by connecting the project leads with potential adopters.

4. The ARTIC program built substantial implementation capacity and leadership across the system.

  • The ARTIC program built a broad range of capacity and leadership in the implementation of research evidence by providing active hands-on implementation experience to >25 researchers, >220 site champions/ coordinators, and >1500 site staff/volunteers.

5. Each of the ARTIC projects reported broader healthcare impacts.

  • The ARTIC projects collectively impacted the care of over 18,000 patients. The lead project teams reported a variety of outcomes related to improved patient care, interprofessional collaboration, patient experience, health outcomes, and system efficiency.

6. The unique value of the ARTIC program lies in its proven model for implementing established evidence-based interventions.

  • ARTIC has proven effective at implementing selected evidence-based interventions in multiple sites consistently, rapidly, and with high levels of fidelity and sustainment. The ARTIC program is unique: there are no other programs in Canada that focus exclusively on intensive multi-site implementation of established interventions.

CAHO engaged Cathexis Consulting to conduct the evaluation and produce a report. The full report, by Cathexis Consulting Inc., is attached for your reference: CAHO ARTIC Phase II Evaluation Final Report