Tuesday, May 25, 2011
8 PDT/11 EDT/16 BST
Attending: Rosalyn Scott, co-chair; Valerie Smothers, staff; Susan Albright, Mary Pat Aust, Simon Grant, Chandler Mayfield.
The minutes were approved.
Rosalyn summarized the key discussion points from the conference around competencies. Many of the working groups, including the two accelerated standards development efforts, are interested in leveraging the competency standard. The biggest discussion at the conference was around the notion of milestones. On the last call we discussed the pediatrics milestones project. We discussed whether milestones should be represented in the standard. After a great deal of discussion at the working group meeting and at several talks, most people felt that the notion of milestones was important, but they felt that it needed to be represented as a separate standard.
Susan commented that she has some confusion around the milestones. It’s not clear when a milestone describes performance and when it describes a subcompetency. If you are trying to achieve X, and there are various substeps, why are they not subcompetencies?
Rosalyn used the Clinical reasoning milestones as an example. The relationship of milestones to the competency is not a hierarchical relationship. They are progressive steps along the way to competence. In addition, they are directly related to assessment. Since we’ve established that performance and assessment are out of scope, milestones need to be treated separately.
Simon commented that there isn’t anything essentially different between milestones and other levels. If we have a mechanism for describing levels, we should make it service for milestones.
Valerie commented that there is an implementation issue. It is feasible to imagine an organization leveraging the same competencies but developing their own framework of milestones or performance levels related to those competencies. That is essentially what is happening in graduate medical education, where each specialty is building on the same set of competencies but developing milestones unique to that specialty. Having the performance framework loosely coupled with the competency framework allows for greater flexibility in the implementation.
Simon commented that it is possible to represent levels as part of a framework. He expressed doubt that inserting milestones separately was necessary.
Rosalyn outlines how milestones may be different for interprofessional competencies. Putting in an IV is a straightforward competency. If you had a health science center that had a set of competencies around that, the competency would be the same for many professions, but the milestones for a phlebotomist will be different than for a medical student. The health science center can have a framework that says everyone in the institution has to put the IV in the same way. But how you get a nursing assistant to that point will different than how a surgeon gets to that point.
Susan asked how we would link levels to a competency.
Simon commented that Tim sees levels as a qualifier to a relationship. You have a thing which you are trying to define levels of, and different things which are levels of it. Those are sub objects. They are narrower in a distinct way. We can represent them by linking up that says what level they are.
Valerie commented that if we proceed with defining a new specification for levels/milestones, we would need to propose that as a new standards development project.
Rosalyn asked Valerie to go over the powerpoint diagram. Valerie explained that the parts in green are the specifications we are currently working on. The parts in yellow are new specifications that need to be written. A separate milestones framework would link to competencies in a competency framework. Additionally, a mapping specification would be required to map various data and resources to milestones and competencies within their respective frameworks.
Rosalyn emphasized that milestones is very much about how you are going to teach this competency.
Susan commented that this is what they have done. The Assessment form puts levels in, they link items with levels to a competency object.
Simon commented that wasn’t different than linking competency objects to assessment.
Rosalyn replied that the way they are being used in pediatrics speaks to a progression of learning. It reflects how competencies are practically being implemented, and what someone is supposed to be able to accomplish at a point in training. Simon commented that how you assess whether they are achieving at that level is a separate matter. Rosalyn agreed. Performance is different than the abstract competency, which is why we are suggesting putting it in a separate but related framework.
Simon commented that the question of defining levels is a different one from measuring levels. A level is what you expect to see, an assessment involves giving people a number, may be number in between. It would be helpful to keep defining and assessing levels as separate.
Rosalyn commented that that is what Tim has done. Milestones are separate from assessment but related. No one suggests that concept of levels is not important. It’s a question of where you place it. We want to keep things as usable for as many different people as possible. In that way, it may be best to deconstruct things that are related so that people can bring them in as they need to. A group that doesn’t think levels are important or doesn’t agree with the levels defined doesn’t have to have to use the levels.
Simon commented that there is no reason in his mind to have a separate spec.
Susan commented that she saw a presentation from Henry Ford hospital regarding their internal medicine milestones. They have defined the ability expected of Internal Medicine residents as they progress through training. That sets the stage for assessment of competence.
Chandler commented that they are implementing milestones and competencies. To students, it is the connection to assessment, but how the competencies relate backwards is also important. It’s like Russian nested doll. What layer are you looking at? It is nuanced as to whether a competency can be observed. In addition, you are never fully competent, you are always moving towards competency. If we don’t have that, we lose something.
Valerie commented that those needs would still be addressed, it is just a question of where.
Rosalyn added that if we can remove milestones from the discussion and move forward with a standard, it will allow other groups to progress as well. Then we can address the notion of levels.
Simon cautioned that there will be consequences of our decisions. At some point it might be useful to prepare a fully fledged example.
Rosalyn commented that the notion of levels is important, but it does not make or break others’ work. Not having a competency framework standard would make or break their work.
Susan agreed it made sense to proceed as Rosalyn had suggested.
Rosalyn asked Valerie what next steps would be in the standards development process.
Valerie commented that the next step would be to draft a charter for the standards development work related to milestones/ levels. The executive committee would need to approve the new charter.
Rosalyn commented that for the next call we would attempt to resolve the outstanding issues related to the competency framework. The next call is Jun 29.
2. Comments from Simon (30 min)
- Requiring a single top competency object that represents the framework
- Location of metadata for a framework
- Location of the type attribute: within the competency object or within the framework association
- Treating a framework as an object
3. Discuss incorporating frameworks in frameworks (10 min)
4. Tasks to proceed in the standards development process (5 min)
5. Open discussion (5 min)