July 16, 2010
8 PDT/11 EDT/16 BST
Please note: the conferencing service will ask you to enter the pound sign. Press # for pound.
To mute, press *6.
Attending: Kim Hoffman, chair; Valerie Smothers, staff; Carol Carraccio, Elaine Dannefer, Maria Esquela, Bob Galbraith, Simon Grant, Linda Lewin, Chandler Mayfield, Amber Montañano, Pat O’Sullivan.
1 Review minutes of last call
The minutes were approved.
2 Discuss educational achievements proposal
Bob reported that most of the group had heard about the idea for a specification around educational achievement. The principal is that any useful portfolio will have chunks of data referring to achievements the learner has made as well as learning experiences. The new specification will address the achievement side. It will be focused around achievements related to the 6 general competencies. Residency programs have a mandate to measure achievement, but measurement has been slow. An educational achievements specification would enable interoperability, allowing educators to build real assessment around the achievement of competence. If you operationalize outcomes, you come up with milestones – expectations of what learner should be able to do. For example, 3rd yr residents in pediatrics should be able to break bad news to parents of a child. The specification might have information, text of the expectation, evidence that resident is putting forward, and judgement of someone with authority as to whether the person has achieved that milestone. It would be generic enough to get get different types of data in as evidence and flexible enough to accommodate milestones for different specialties.
Carol emphasized that this work was incredibly important. At the ACGME milstones meeting last week, it was evident that people are in very different places. There may be commonality over time. There are major differences between medicine and surgery. The specification should support commonality where it makes sense and be flexible where it makes sense.
Bob added that a potential benefit might be that it could assist in herding the 24 specialties in a common direction.
Lindsey questioned whether this is far enough ahead from others that they will decide life is too short for tangents? Or will there be engagement across different groups?
Bob replied that we’ve shown with educational trajectory we could put something together in 10 months, which is an incredibly short time to design a complex specification. We can do it again. If we can come up with something that makes sense in 10 months, we’ll have made major contributions. It would take years to build a competing spec. The ACGME is in awkward position. The deadline has passed, but not much measurement is happening. If the specification is easy to use and broad, it would make a major contribution.
Simon agreed it was a fantastic idea. A lot of people will be really interested. Another bucket may be useful – method of assessment. In many cases, this is an expert judgement. But that is not always the case. Bob agreed.
Carol commented that pediatrics is trying to capture this idea. The milestones group would be ready willing and able to consider this. They have some thoughts already on how they might assess milestones. That would dovetail well with what we are talking about.
Bob commented that we need to expand the working group to include those who have a residency perspective and those who can speak on behalf of the boards. We probably can’t have every specialty involved. Pediatrics is an obvious choice; they have done a lot of work already. Internal Medicine and Surgery are important as well.
Lindsey commented that at Minnesota Orthopedic surgery and neurosurgery have been ahead of the curve. Elaine added that neurosurgery s out in front at Cleveland as well. Their professional association has come out with technical skills for residency.
Maria commented that Cardiology and GI have had many years trying to establish quality indicators in patient care. These are for use during and after training.
Linda asked if ACGME should be involved. Bob replied that the question is how. He doesn’t know yet. The ACGMe provides accreditation for residency. From his perspective, we need representatives from program directors organizations, specialty boards, and the ACGME. The group agreed that sounded reasonable. Bob agreed to put together a list of individuals to solicit. He will then come to this group with that list. There will be some lead time in getting up to speed.
Kim asked about the sponsorship for the project. Bob indicated that the AAMC and NBME are both willing to sponsor. It is unclear if the federation will be involved.
Report on GEA Meeting
Bob reported that AAMC group on student affairs (GSA) and group on educational affairs (GEA) had a combined meeting and spent 2.5 hours talking about portfolios.
Pat elaborated that there were about 50 people, 15 from GEA. There were a lot of questions, mainly concerning legal issues. They are also concerned about protecting registrar duties. The audience struggled with why would you do this, how could you do this for 20 years since data is purged? Bob added that many people were engaged in thinking about what else could we do with real time data.
Valerie explained the changes made to the way publications are represented. She incorporated all of the group’s feedback from the last call, including the decision to allow a free text description of the role, whether or not the publication is peer reviewed, whether it is a publication or presentation, and the description of presentation.
Pat commented that case report seemed an inappropriate example for presentation. Kim agreed. The group decided to remove that from the description of publications.
4 Open discussion
Kim asked Pat about the report to the GEA. Pat replied that she did take the report to the GEA. The GEA seemed happy with the report. There was no action for them to take; they did work to understand the pilot and what was accomplished.
Case report will be removed from the description of publication.
Bob will put together a list of individuals to solicit for working group participation.