April 3, 2012
8 PDT/9 MDT/10 CDT/11 EDT/16 BST
Attending: Linda Lewin, co-chair; Carol Carraccio, Kelly Caverzagie, Bob Galbraith, Patty Hicks, David Melamed, Sandhya Samavedam, Valerie Smothers, Sandra Waters.
1. Review minutesof last call
The minutes were approved.
2. Discuss revised powerpointpresentation
Valerie explained that the specification brings together trajectory, curriculum, competency, and assessment data. The curriculum, trajectory, and competencies serve as lenses through which you can view the assessment data. Carol recommended addingd epas to the competencies circle. It’s a different perspective. Sandra recommended darker lettering for Curricula and Educational trajectory. Valerie updated the slide deck and adjusted the date.
Patty commented that Curriculum isn’t linked directly to assessment data, but to competencies. She asked if EPAs represent something larger, at a more overarching level. If one wanted to generate assessment data, would it be linked to a specific curriculum?
Valerie commented that the design she is proposing would have all assessments reported in conjunction with a curricular element (a course, clerkship, etc). Since competencies are also linked to curricular elements the assessment data could also be viewed by competency, with the curricular element within which the assessment occurred noted. Organizations like NBME that provide assessment only would be reported through a curricular element listed as “NBME” but could still be linked to a competency (medical knowledge).
Valerie described the second slide, which shows how the technical specifications are interrelated. Linda clarified that EPAs would be represented using the Competency Framework.
Carol asked if we report more granular data about milestones, which programs will have to do, will there be a mechanism that shows how that data flows into the appropriate competencies? Valerie replied that the Performance framework specification would delineate the connection between milestones and competencies to show that data flow. She asked Carol is she had a sense of what the data the specialty boards would be collecting given the new accreditation program from the ACGME. Carol commented that the Residency Review Committee (RRC) is meeting in mid April. She would have a better sense of where things are going after that.
Kelly commented that he anticipates playing a part in those conversations in Internal Medicine. They are convening stakeholders sponsored by the board to lay out a roadmap. They need to determine how the RRC and ACGME describe what they want. Conceptually, they are close to the same page.
Valerie described the other changes made to the slide deck. The 3rd slide shows all assessment data being shown as curricula, and changes to the language around activities which are now called Entrustable Activities and Skills. Those changes are propagated throughout the document. Slide 13 shows the detailed information that would be available about courses within the curriculum and how that data would link to the definitions and assessment of competencies, as well as the parent course within the curriculum. On slide 16, the entrustable activities shows the related curricula and where in that curricula the activity was assessed. The methodology on slide 17 uses a value from the list of assessment methods within the curriculum inventory specification.
Patty commented that this type of skill would have to be directly observed. She recommended changing the methodology. Most likely it would be the supervising resident or faculty member observing. Valerie agreed to change the methodology.
Valerie continued with the updates on slides 26-28. The educational trajectory would also be able to link back to the appropriate part of the curriculum. Slides 30 through 35 show the curriculum the learner has experienced at University of Toronto. There are links back to the competency framework and the relevant assessment data. Slides 37-30 show assessment data from the NBME and ABP respectively. The data is shown as a “curriculum” of assessment.
Patty asked about the grid on slide 26 and whether the use of quarters would be uniform. Valerie explained that exact dates are used throughout. The slide shows one way of displaying that data, but any useful increment could be used.
Patty asked if the EPAs could be time limited. Are there epas that have an extinction date or need to be done at a specific frequency? Could you review someone’s transcript know what is current and what is not?
Valerie commented that Rosalyn Scott had raised the same question at a recent meeting Co-chairs retreat. At that time the discussion was that it might be premature to include that data as we don’t have extinction dates or frequency requirements currently.
Linda commented that a lot of things like board exams that will be included have to be redone every so often. But it’s not the job of this system to prompt you that this is expired. It just tells you when this was done. Someone needs to figure out whether or not things are current.
Patty commented that she is thinking of convenience. If she were looking at an applicant that had accomplished ACLS certification more than 2 years ago she would want to see a notice indicating that it is expired certification. That relates to the way data is stored, that it is timelimited. There is usefulness in reporting that data.
Valerie commented that the professional profile includes things like board certifications and their expiration dates. Certification is more related to credentialing. That should be thought of as distinct from educational achievement.
Linda commented that my entrustment is like certification. She could see there being language that this is good for 2 years.
Valerie commented that there needs to be clear distinction between the various specifications. If the certification is time limit, we would leverage one of the existing pieces in the professional profile. Valerie agreed to work on a draft specification for next time.
3. Discuss draft data model
4. Open discussion
David commented that it is interesting to see the cross reference to competencies and assessment. GME and UME are so vastly different. It will be interesting to see how we gel these together over time, especially with regards to EPAs. In UME, the curriculum is more important to find uniformity and progression. With residents, it’s about defining epas, making sure they reach them and sustain them.
Valerie will post an updated slide deck
Valerie will draft a specification