November 20, 2009
8 PST/11 EST/16 GMT
Please note: the conferencing service will ask you to enter the pound sign. Press # for pound.
Attending: Kim Hoffman, Chair; Brownie Anderson, Michael Barbouche, Karen Beggs, Carol Carraccio, Elaine Dannefer, Bob Galbraith, Gwen Garrison, Maureen Garrity, Simon Grant, Linda Lewin, Chandler Mayfield, Neil Mehta, Valerie Smothers (staff).
1 Review minutes*of last meeting*
The minutes were approved.
2 Review Educational Trajectory Update summary of progress
Kim commented that the educational trajectory update PowerPoint summarizes major decisions to date. It's designed to be a resource for those who are new to the group or those who have missed calls. Kim commented that the presentation would be kept up-to-date and encouraged the group to send any comments to Valerie or Kim.
3 Discuss feedback from AAMC meeting
Kim summarized that several group members have presented a workshop at the AAMC meeting. In addition, group members presented a poster on the project. The feedback has been good; Marianne agreed to synthesize feedback and circulate to the group. Kevin's notes on the feedback are currently posted to the wiki. Bob commented that the document captured the discussion well. Chandler added that there was lots of discussion verifying the data we planned to collect.
4 Discuss proceeding with small groups
Kim explains that to move forward more quickly we will have small groups that focus on a particular topic and report back to the main group for further refinement. Several individuals have stepped forward and offered to lead the small groups:
- academic difficulty and leaves of absence, Pat O'Sullivan
- formal extracurricular learning, Elaine Dannefer
- informal extracurricular learning, Linda Lewin
- technical architecture, Kevin Souza
The small groups will go through a particular area and answer the types of questions included in the specification excerpt which was circulated earlier. Questions are related to completeness of data, requirements, examples, and feedback. On the main calls we will look at interactions between the main categories and refine our approach.
Linda asked if we would change the schedule of the larger group so that the small groups could speak at the same time. She commented that it would be difficult to schedule meetings for the small groups.
Kim replied that we could have reporting from small groups at strategic times and not use a full hour for the calls.
Pat asked what the time frame was for establishing the small groups. Kim replied that we would establish them as soon as possible. She asked the group to continue the conversation via the e-mail list or wiki.
5 Review timeline*for pilot project*
Kim commented that Valerie had provided a timeline for a pilot project being undertaken by several of the medical schools involved in the working group. Michael and Bob had suggested some changes to the timeline and the technical approach. She asked Valerie to describe their discussion and the changes that had been made.
Valerie summarized that the time line showed the steps involved in implementing the specification for purposes of the pilot. The changes that had been made are designed to make it easier for schools to send data and lower the barrier to participation. Instead of having a data repository outside of the firewall, the pilot could allow participants to send an Excel spreadsheet or XML document to a central secure website at the NBME or AAMC. This eliminates some steps and reduces technical barriers while accomplishing the end goal.
Neil asked if someone would provide an Excel template to facilitate the development of the spreadsheet. Valerie commented that she would prepare such a template as soon as we had an initial draft of the complete specification.
Simon commented that the technical group should decide the format of the specification before detailing other parts of the specification. He added that an Excel spreadsheet would not be able to express cross-linked data the way that XML can.
Michael agreed and emphasized that it would be beneficial to proceed in phases. We could learn much from getting schools to send data in a structured Excel format for phase 1 and then get at richer data at a later time. The needs of the pilot need to be balanced with the resources and political will of those who are participating.
Simon commented that it would not be too difficult to get people to export leap 2A data.
Kim asked for medical schools to comment on the timeline. Getting access to resources may be a challenge in December.
Linda agreed that beginning the export between December 18 and January 8 would be very difficult. After December 18 most people are on vacation. They come back on January 4, so the January 8 deadline is probably not realistic. Pat added that California is on mandatory furlough one of those weeks.
Bob commented that the timeline is more of a guideline.
Simon commented that it may be a good idea to put technical staff responsible for the pilot together with the technical architecture small group. That would allow for a dialogue as to what will take how long.
Kim commented that the Excel spreadsheet would provide a low barrier to entry since they routinely import and export data using Excel.
Neil commented that many schools will have some data that they will have to put in manually. Unless a programmer is available, they will want to use Excel because that can be completed by an average administrative person. Chandler agreed, commenting that there would be some holes that would need to be filled in manually in their system. Simon questioned whether learners could input the data. Bob commented that there would not be verification with learner-provided data.
Kim added that verification was discussed in the workshop. There was a concern that medical schools need to take more responsibility and verifying extracurricular activity details and capturing the intensity of the individual's participation in that activity.
Bob added that self reporting could be a part of later efforts to implement the specification. The pilot is intended to test whether we can pass data from school to school. For the purposes of feasibility test, self reporting is not something we need to test. In addition, getting students to self report data would further extended the timeline. There is a notion of the timeline of data coming to a central repository. For the short term, that is a feasible approach. The NBME has resources to bring to bear that would allow for a central secure website to be developed quickly. Individual schools may not have the resources for other approaches. Gwen agreed.
Chandler asked if data would be identified. Bob replied that it would. NBME currently gets most of the data for MSLE sign up. Chandler commented that there are some additional fields. Neil asked if hypothetical data could be used if Deans object to using identified data. Bob commented that it would be possible but that it would not be a proper demonstration. If people are unwilling to share data, educational trajectory exchange won't happen. Neil commented that if Deans see the final product they may change their minds.
The group commented that a one page document to explain the security of the data and the efforts of AAMC and NBME would be helpful. Brownie commented that she could work with the Council of deans to draft a letter to those participating in the pilot. Bob added that he has drafted a letter. Valerie added that she has drafted a summary document which could be leveraged as well. Valerie agreed to send her summary to Bob and work together to come up with a common approach.
6 Open discussion
The next meeting is Dec 4. We'll figure out more of an agenda for that meeting with group leaders.
Dec 18 Kim is not able to join; we will think about how to use that time.
- Valerie will prepare a description and technical overview document to accompany the timeline.
- Kim will update the PowerPoint presentation to include the leaders of small groups
- Valerie will circulate a mechanism for members to self select participation in small groups
- Valerie and Kim will schedule calls with the small group leaders
- Brownie, Bob, and Valerie will work to share their drafts and explore processes for communication
- Brownie will contact the Council of deans regarding a letter to participating schools