2011-08-23

Meeting Information

Please note: the conferencing service will ask you to enter the pound sign. Press # for pound.
To mute, press *6.

Attending:  Linda Lewin and Alan Schwartz, Co-Chairpersons; Valerie Smothers, staff; Susan Albright, Dana Bostrom, Rachel Brown, Sharon Coull, Mike Dugan, Robert Englander, Maria Esquela, Bob Galbraith, Maureen Garrity, Simon Grant, William Iobst, John Jackson, Patty Hicks, Chandler Mayfield, Sandhya Samavedam, Howard Silverman, and Loreen Troy.

Agenda Items

1 Review minutes of last meeting

Dana introduced herself to the group as the new AAMC portfolio manager. She is working jointly with NBME to develop an eFolio connector.

Simon had a revision for the sentence on the higher education achievement report, changing it to “A would be important in Europe”.  Valerie made the change and the minutes were approved as amended.

2 Summary of professional profile

Linda explained that an overview of some existing standards would be helpful so that the group better understands what needs to be developed and what is already done and does not need to be captured in the educational achievement standard. Valerie provided an overview of the Healthcare Professional Profile standard.  The professional profile describes the individual physician; the standard has been out for a few years and has good adoption.  It includes unique identifiers, name, address, education, training, certification, licensure, disciplinary action, academic appointments, occupation, personal information and professional membership.  Valerie noted the certification for licensure is US specific. The standard has been implemented by ABMS, ABP, the AAMC, and the NBME using it as part of their data commons effort. 

Simon asked how much detail there is in each section.  Valerie commented it’s pretty detailed.  For example, the section on licensure includes contact information, notice of actions, appeals, and judgments, and each of those areas contains more details.  In the education section, there is degree, information about the institution, distinction associated with the degree, status of the learner (active, graduated, leave of absence, etc.), start and end dates, and more.  Linda asked the group if they will want more detailed information included in our standard.  Valerie commented it is hard to say what other people want; however; she can envision boards wanting detailed information about USMLE results and resident competencies as part of the initial certification or licensure.  Bob agreed we need all that data to check all the boxes that have to be checked.  He mentioned that state licensing boards get nervous when there are time gaps; educational trajectory would be helpful in documenting those.  Bill mentioned the ABMS issue of how flexible entry would be, and whether the data would be considered primary source information and the willingness of the board to accept data coming in.  Mike Dugan from the FSMB agreed with Bob that the USMLE is important to all 70 jurisdictions, and residency gaps or unusual circumstances need explanations.  Primary source verification is the bigger question.  He asked if the professional profile includes USMLE scores.  Valerie replied that it does not include assessment details or statements of competence; it’s more about credentials. 

3 Discuss potential users for each use case and identify priorities if applicable  

Valerie continued with a description on identifying users for each use case and prioritizing if need be. We want working group members to be champions for particular types of users so all perspectives are well represented.  The learner is at the center, and the learner using their own data is first priority (use cases 1, 6 & 7). The researcher (use case 3) is another user.  Administrator (program director, dean, continuing education director, use case 4) looks at data for the whole group of learners for program evaluation.  Use case 5 is the mentor, and finally we have a number of external reviewers identified (use case 2).   External reviewers include credential designating organizations like licensing boards, certifying boards, hospitals,  professional organizations or associations, and insurance companies.  There are also education and employment related users.

Linda asked if any groups are missing.  Maria mentioned possibly adding FEMA to the list because disaster recovery people have to be credentialed. She mentioned the NIMS national credentialing system.  Linda asked if that would fall under certification boards. Maria agreed to send a link to the group for review.  Linda asked if the promotions committee would fit under use case 2 for external reviewers and if there were other credentialing organizations for Europe that we need to have on there.  Sharon replied that she couldn’t think of any others.  Linda questioned the inclusion of promotions committee and peers.  Bob suggested that in the future we may be providing clinical data back to the physician in practice; they may want to send a data set to peers to look at.  Linda agreed that made sense. 

Linda then reviewed the employment related reviewers. Bob asked if employers would handle privileging.  Valerie noted that hospitals were listed as a credentialing organizations for that purpose.  Patty commented achievement data would be helpful for search committees reviewing status prior to the credentialing process.  Linda suggested changing it to maybe selection/search committee. 

Linda asked the group if they were willing to volunteer to be champions for specific types of users.  That would allow us make sure all perspectives are represented and ensure the specification meets the needs of those users.  Valerie provided an example: if Alan is a researcher looking at data and doing research, he would want to make sure he can aggregate data using the standard.  Linda noted we don’t have a student or resident learner, and it is important to identify somebody and get them in the group.  Howard suggested we could create a process to have student representatives from AAMC and have representatives on the calls.   Rachel mentioned the Organization of Students Representatives has a national Chair and steering committee that are active.  They have 2-3 representatives per school and meet twice a year, and would be happy to be involved.  The steering committee chair is Joseph Thomas. 

Bob commented there are 4 student organizations and we should have a feedback loop to all 4 groups.  The four groups are American Medical Student Association (AMSA), OSR, AMA Medical Student section, and the Student National Medical Association (SNMA). Linda suggested identifying a contact person and sending them summaries asking for feedback.   Bob mentioned having a formal process for getting feedback from outside the US in Canada and Europe; that would show us making an effort to obtain not just US input.   Valerie agreed it would be a good idea to have a formal process to get feedback. She suggested identifying a contact person in each organization, presenting some background on the work we are doing, so that person can understand the context and share that with colleagues, and continuing to give them updates.  When we are ready with the first version, we can share that with them.  It is hard for people to wrap their heads around specifications; additional explanations are helpful.  Maureen suggested Ali Anderson at AAMC could provide the leadership names.  Bob said the NBME is in contact with all four student groups, and they are familiar with leadership.  Valerie agreed to coordinate with Bob to get points of contact for student organizations in the US and provide them with background and a request for ongoing updates and specifications review.

Bob asked if there was a similar organization in Europe we could think about.  Sharon said she would try to find out and agreed to look at the British Medical Association (BMA) and European Medical Student Association (EMSA); they host a lot of events.  Linda suggested having an update on the next call regarding student and resident representatives.  Bob commented it is important we do get input from students and residents and make it clear that we are working in this area; it spreads the words of what we are trying to do and makes it clear that we are trying to be transparent about all of this.  Linda agreed.

4 Identify champions or ask for volunteers

Linda asked whether we need to ask people to sign up for areas or have it be less formal?   Valerie commented that iIdentifying people in each category will allow us to come back to specific points and check in, and that could be useful.  Valerie offered to send out a request via email.  Mike volunteered to represent licensing boards.  Linda will provide the medical school perspective, and Bill volunteered for to give the certification perspective.  Rachel said she could provide the non-academic practice view. 

5 Request examples of educational achievement data

Valerie explained that we are wrapping up the use case phase and going into a data analysis phase to look at what people are currently collecting in terms of education achievement data.  She suggested sending in examples of educational achievement data to be considered.  We can compile the results and discuss on the next call.  Linda clarified that shelf exam scores and results of observed clinical encounters would be examples.  Lori shared that each medical school develops a Medical Student Performance Evaluation (MSPE or Dean’s letter). That has many common elements and would be an important place to start.  Valerie asked if that is at the level of individual competency. Lori responded it follows national guidelines and includes such items as matriculation date, basic science courses, whether there honors were achieved, clinical years, academic achievement in terms of clinical evaluation, knowledge assessment, and final composite score for clerkships.   It very inclusive and has comments regarding students performance, and a narrative component as well.  Students have personal comments to describe themselves.  Howard agreed that the Dean’s letter is universal and a summary of record.  Maureen commented the GSA has a task force we could hook up with.  Valerie agreed that the Dean’s letter has valuable information but we don’t want to limit and look at it as a launch point for moving forward.  Maureen mentioned the GSA groups is also looking for innovations, and it would be a good partnership.  Patty shared Emergency Medicine published a standardized language to assist in the scoring process.  She thought that was worth looking at.  Linda agreed we should look at both MSPE reports, clerkship comments, honors, and whole rich oral exams.  There are a huge number of items to get at, and she suggested starting with the MSPE’s and what they are actually made of.  She asked the group to send examples of things people collect to Valerie and she will compile a list. 

6 Open discussion

 Nov 8 call cancelled.

Decisions

Action Items

• Valerie will coordinate with Bob to get points of contact for student organizations in the US and provide them with background and a request for ongoing updates and specifications review.
• Valerie will send out a request for champions via email
• Group will send Valerie educational achievement examples
• Sharon will find out information on medical student organizations in BMA and EMSA and look for a point of contact.