Medbiq_logo_rtm.gif

Performance Levels and Framework Standards Development Proposal

 

Document version: 0.1
Date: 27 May 2011
Author: Valerie Smothers (vsmothers@jhmi.edu)

Goal

Competency-based learning and assessment is increasingly being applied within health professions education at all levels across the continuum. An important component of competency-based learning is the concept of milestones, expected levels of performance related to a competency. In the US, the Accreditation Council for Graduate Medical Education (ACGME) has convened working groups to develop Milestones for each of the 26 specialties in its purview. In undergraduate medical education, several medical schools, including the University of California, San Francisco, are implementing milestones related to the developmental achievement of competence. In other health professions, levels of performance are frequently associated with the assessment and development of competence. One example is the Association of Critical Care Nurses Synergy Model for Patient Care, which specifies core competencies for clinical care nurses and levels of expertise ranging from competent to expert.

Currently there are no technical standard for milestones/performance levels or frameworks describing the relationship among competencies and performance levels. A common technical format for performance frameworks would facilitate the dissemination of performance frameworks and their implementation in curricula and performance assessment systems. Technical standards would also make it easier to connect assessments to performance and competencies. Other standards development efforts would require the existence of a performance level and performance framework standard. In particular, the MedBiquitous Educational Achievement work, designed to document learner competency and make that data portable, would require electronically citable performance levels. Without standards, performance data is not portable or easily understood across the continuum of a learner’s career.

Context

MedBiquitous develops information technology standards for healthcare education and competence assessment. Through Working Groups and a Standards Committee, MedBiquitous members are creating a technology blueprint for healthcare education and competence assessment. Based on XML and Web services standards, this blueprint will weave together the many activities, organizations, and resources that support the ongoing education, performance, and assessment of healthcare professionals.

In 2008, the ACGME launched its Milestones project to articulate expected levels of performance at entry to practice and developmental milestones to ensure that residents remain on track to achieve their educational goals ( ACGME Bulletin, May 2008 ). ACGME committees in concert with leaders from ABMS-member specialty boards are defining milestones for each specialty.

In parallel the Association of American Medical Colleges and the National Board of Medical Examiners are moving forward with plans to develop a system to compile learner data to enable the learner’s ongoing growth and develop. This eFolio Connector is envisioned as compiling data from e-portfolio systems at individual medical schools as well as other sources of data relevant to the learner. Performance data related to the achievement of competence is within the scope of this project. A Performance Framework specification would contribute to the growing body of work enabling interoperability of eFolio data.

Standards Environment

HR-XML provides technical specifications for employee performance appraisals, referred to as EPMResults (Employee Performance Management is abbreviated EPM throughout their specifications). The specification provides a detailed model for communicating the results of these assessments, including an appraisal reason, overall rating, results related to specific objectives and competencies, information about the raters, and remuneration recommendations.

IMS Learner Information Package provides technical specifications for listing competencies that have been attained by a learner as well as assessment results, goals, qualifications, activities, interests, accessibility information, transcript, affiliation, and relationship.

The MedBiquitous Competency Object specification and the MedBiquitous Competency Framework specifications will likely play an important role in the development of technical standards for performance levels and performance frameworks.

None of these is designed to enable the articulation of performance levels related to specific competencies.

Scope

We propose developing requirements and XML data specifications for the exchange of performance levels and performance frameworks. The initial focus of the working group will be on undergraduate and graduate medical education. North American and International paradigms for medical education should inform the structure of the specification as should performance assessment of other healthcare professionals.

Whenever possible, the group will leverage useful specifications developed by other organizations. The MedBiquitous Technical Steering Committee will offer guidance and technical support when needed.

The specifications and services created by this working group may serve as foundation pieces for other specifications designed by MedBiquitous and will be architected to allow for other parts of the MedBiquitous blueprint for healthcare education and competence assessment.

The working group may develop guidelines to provide guidance to healthcare educators wishing to implement the standards. It is expected that the working group will further refine this scope outlined in this charter to best meet their goals.

Work Plan

Working Group members or staff will perform much of the group’s work independently with member comments submitted to a discussion list and documents shared via wiki. While there is no timeline for development, it is expected that development will be facilitated by a wealth of literature on the structure of milestones and milestone frameworks.

References

Association of Critical Care Nurses Synergy Model for Patient Care. Accessed May 27, 2011. http://www.aacn.org:88/wd/certifications/content/synmodel.pcms?pid=1&&menu =

Robert Galbraith, Portfolios for Lifelong Competence Assessment, May 11, 2011. http://www.medbiq.org/sites/default/files/presentations/2011/Galbraith_2011.pptx

Thomas J. Nasca, “The CEO’s First Column — The Next Step in the Outcomes-Based Accreditation Project,” ACGME Bulletin, May 2008: 2-4. Accessed May 27, 2011. http://www.acgme.org/acWebsite/bulletin/bulletin5_08.pdf  

Exploring HR-XML 3.0 Release. Accessed May 27, 2011.
http://www.hr-xml.org/resources/2009-10-15_Exploring_HR-XML_3_0_Schema.pdf

IMS Learner Information Package. Accessed May 27, 2011. http://www.imsglobal.org/profiles/lipinfo01.html

MedBiquitous Competency Framework. Accessed May 27, 2011. http://www.medbiq.org/working_groups/competencies/CompetencyFrameworkSpecification.pdf

MedBiquitous Competency Object. Accessed May 27, 2011. http://www.medbiq.org/sites/default/files/CompetencyObjectSpecification.pdf

MedBiquitous Educational Achievement Project Proposal. June 10, 2010. http://www.medbiq.org/sites/default/files/EducationalAchievementsProposal.pdf