February 29, 2012
9 PST/12 EST/17 GMT
Attending: Rosalyn Scott and Tim Willett, co-chairs; Mary Pat Aust, Simon Grant, Valerie Smothers.
1 Review minutes
The minutes were approved.
2 Discuss others to include
Rosalyn commented that our numbers have gone down. People may think our work is over; they are not aware we have a new project. She asked who else we should include.
Valerie commented that Bob Galbraith had recommended Steve Clyman from the NBME. He may be able to suggest others, too.
Rosalyn asked if there were suggestions for people outside of the US. Simon asked Rosalyn to clarify what sort of people we are after. Rosalyn replied individuals interested in how we are assessing people. Valerie commented that those actually using performance frameworks for assessment would be most desireable.
Simon commented that he know some people from the medical establishment. He also knows people more generally from the skills area in Europe. Most of the health competencies tend to be yes or no. The Royal colleges do have professional exams, but they are not very technically aware.
Rosalyn asked about representation from Singapore.
Valerie agreed National University of Singapore would be a great addition. They are a regional and global leader in medical education. They host a large medical education meeting. They have both a British style and an American style curriculum. Josh Jacobs from Hawaii now works there. He participated in our curriculum inventory working group and presented interesting work. Valerie agreed to ask Josh about participation.
Rosslyn recommended seeing if we can identify someone from a specialty other than pediatrics. Valerie recommended Kelly Caverzagie in Internal Medicine.
Simon commented that Andrew Lamb from the Royal College of Surgeons of Edinburgh would be a good addition. He was the lead in the portfolio interoperability project. They have launched a surgeons portfolio that deals with revalidation. Valerie agreed to ask Bob Galbraith for an introduction.
Tim commented there are a couple others we should look into based on the literature.
Krackov and Pohl from Albany Medical College have an article on establishing milestones for summative assessment.There’s another on competency training in internal medicine from . Steven E. Weinberger, Anne G. Pereira, MD, MPH; William F. Iobst, MD; Alex J. Mechaber, MD; and Michael S. Bronze, MD. Beatrice A. Boateng, PhD, Lanessa D. Bass, MD, Richard T. Blaszak, MD, and Henry C. Farrar, MD have published on the development of competency based assessment rubric. Tim will send abstracts for all three. We can consider approaching those authors.
Rosalyn recommended approaching someone from emergency medicine. That specialty is procedurally oriented. They have been looking at competencies for a long time.
Tim asked Mary Pat about Nursing. Mary Pat suggested maybe folks from Healthstream. They have a competency framework product they use. They may be able to shed some light. Valerie agreed to follow up with Spenser Aden.
Simon commented that the Royal college of nursing is involved in portfolio work. Their knowledge and skills framework may have levels. That could be a worthwhile avenue to explore.
3 Updates to competency definitions, AAMC and emerging list on wiki
Tim explained that on last call we spoke with Bob Englander and came to some common understandings. The linked page is a combination of AAMC definitions and ours. Nothing is a conflict. There are a few definitions we haven’t worked with. One is performance level. There is a new definition for performance criteria.
Valerie commented that competency object should be defined.
Rosalyn questioned the difference between epa and a performance framework.
Tim replied that there is a lot of overlap. An EPA is a thing a professional could do. One must demonstrate certain levels of performance before being allowed to do an epa. Rosalyn replied that programs are using epas to identify things trainees can do independently. She suggested EPA is another kind of competency. Valerie agreed, noting one additional factor is that an EPA rolls up several competencies from different domains. Simon commented that it is its own hierarchical tree and is not qualitatively different.
Rosalyn commented that looking at Ann Burke’s paper, the concept of entrustment is that a practitioner has demonstrated necessary knowledge, skills, and attitudes to independently perform an activity. It is integrating the competencies.
Simon commented EPAs will overlap in the competencies they require. The EPA is where it comes down to practice.
Tim commented that EPAs are like a list of competencies necessary to achieve. But EPAs don’t go so far as to indicate what behaviors need to be demonstrated. There is also the notion of nested EPA. Nested EPAs may need to be accomplished first.
Rosalyn asked if EPA requires a separate XML from the competency framework. Tim agreed that is the bottom line question. Rosalyn added that EPAs are also important for assessment. Rosalyn and Valerie agreed to see if an EPA can be captured using the existing competency framework specification.
The group discussed the notion of performance levels. Simon commented that levels can be generic or specific to a competency.
Mary Pat commented that the concept of performance levels is consistent with their competencies in the synergy model. There is a description of levels 1-5 (novice to expert) and what that looks like. But not specific criteria of what is necessary to be considered an expert in clinical judgement.
Tim commented they are looking at performance framework as an extension of competency framework that gets into more detail. It could be a generic schema; we need to distinguish that from a performance framework as an extension to a specific competency framework. Tim offered to revise the definitions.
4 Review/brainstorm use cases
5 Identify examples for review
6 Open discussion
- Invite the following people to participate
- Steve Clyman
- Josh Jacobs
- Kelly Caverzagie
- Andrew Lamb (ask Bob Galbraith)
- Healthstream (Valerie will contact Spenser Aden)
- We’ll work to identify representatives from
- The Royal College of Nursing
- Emergency Medicine
- Authors of relevant papers
- Rosalyn and Valerie will model an EPA as a competency framework.
- Tim will revise the definitions.