Competencies Working Group, MedBiquitous Annual Conference, May 9, 2011
Attending: Rosalyn Scott and Tim Willett, co-chairs; Valerie Smothers, staff; Susan Albright, Ian Graham, Lise McCoy, Parsh Mehta, Juliane Schneider, Litsa Mitsopoulou, Luke Woodham, Peter Greene, David Stumpf, Jeannie Poterucha, Nabil Zary
Rosalyn asked Valerie to begin by describing the working group process. Valerie explained that new projects must be approved by the MedBiquitous Executive Committee. The bulk of the work to develop a standard takes place within the executive committee, and then the candidate standard is sent to the Standards Committee for further review and balloting. Any issues raised must be resolved by the Standards Committee in concert with the working group.
Rosalyn added that we started in the group with use cases that describe how we would use the standards in practice.
Tim added that we can no longer think of this as a standalone working group. The competency framework intersects with many other standards development projects in MedBiquitous. We need to consider how it relates to those other activities. As part of the group’s work, we conducted a survey asking about the use cases. The most important use case was Record performance data as evidence of competence. Full results of the survey are available at: http://groups.medbiq.org/medbiq/display/CWG/Survey+analysis
Using these use cases, the group developed a conceptual model of how competencies intersects with performance data, learning content, and other items. The conceptual model is available online as: http://groups.medbiq.org/medbiq/display/CWG/Conceptual+Model At the heart of the model is the notion of a competency object – an individual statement of expectation, wither a competency, learning objective, learning outcome, etc. Competency objects may them be assembled into a cohesive whole using the Competency Framework specification, which defines how competency objects related to one another.
David asked if the competency object category had multiple axis – would it get into skills vs knowledge? Tim explained no; the definition leaves it open to others to define. Tim explained that the group explicitly excluded assessment methods, performance criteria, level, degree of competence, conditions of performance, context, outcome criteria, and relationships from the competency object. The competency framework intentionally omits external relations and mapping to other frameworks.
Litsa asked how it related to rdf.
Tim replied that the types of relationships described are narrow than, broader than and related. The binding used is XML. The group has determined that we also need levels; what do I expect for performance as the learner moves from novice to expert? Those are Milestones. Achieving a competency takes a number of steps; the learner is expected to move through those steps to achieve the competency.
Rosalyn commented that the notion of modeling a milestone as a competency object was controversial. We had previously conceived of milestones as related to performance, which was intentionally omitted.
Susan commented that the milestones were very similar to the grading scales used by her institution.
Tim summarized that what the group was considering was a reference within the competency framework that indicates the relationship of milestones to competencies. There would be an ordinal component to the relationship.
Ian commented that he was troubled by embedding performance in a competency framework. Competence and performance are two different axes.
Tim commented that for a single competency, such as suturing, the expectation would be different if applied at different levels fo medical education.
Rosalyn commented that assessment should be separate from the competency framework. A level has to be described as performance or it has no meaning.
The group reviewed the pediatrics milestones project (see http://tinyurl.com/5wcjwzg) and the milestones related to clinical decision making, noting that it is a progression towards competence. Valerie explained that for other milestones, such as Truthfulness, the progression was quite different, in that one end of the scale was lying and the other end of the scale was open communication and acknowledgement of mistakes. A learner could easily begin a program and not be a liar.
Ian warned about muddling performance and competence. The milestones, he argued, are clearly linked to performance. David agreed, saying that at United, all they measure is performance.
Nabil asked what would happen in a multi-national effort where clinical reasoning is defined quite differently.
Tim commented that one could reuse the competency statement and change the levels.
Parsh asked if milestones were represented as requirements.
Luke commented that competencies and milestones are different things. You can’t change the competency object, but you might change the milestones around it. Modeling the milestones as a separate entity gives you that flexibility.
Peter added you could map assessment events to milestones using a separate milestones framework. Is that a part of a performance framework? Tim commented it would still contain statements represented as competency objects connected by relationships The data model would be very similar.
Peter asked if the performance framework would have assessment objects in it. The group left that question open.
Rosalyn wrapped up the meeting telling the group that we welcome ongoing participation. Email Valerie and tell her if you would like to continue.