Educational Achievement
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

Meeting Information

Date:

June 28, 2011

Time:

8 PDT/11 EDT/16 BST

Attendees: Linda Lewin and Alan Schwartz, Co-Chairs, Susan Albright, Rachel Brown, Bob Galbraith, Simon Grant, Patty Hicks, Bill Iobst, Morgan Passiment, Howard Silverman, Kevin Souza, Andria Thomas, Jan Trial.

Agenda Items

1.       Introductions

Linda began by announcing that she and Alan Schwartz are the group’s co-chairs and would be facilitating the meetings.  She then began the introductions, starting with herself.  Linda is a general pediatrician at the University of Maryland and she is involved in the e-folio advisory group, or EAG. She has worked previously with Valerie on the Educational Trajectory Project. Alan is the Director of Research in the Department of Medical Education at the University of Illinois in Chicago.  He is actively involved in several projects related to assessment of educational achievements in Pediatrics.  Valerie is the Deputy Director of MedBiquitous and she staffs all the working groups and drafts specifications and provides the technical support to achieve the goals of the group, along with Jody Poet who is taking the minutes of the meeting.  Simon Grant is from England and he works with JISC, the educational technology and standards based group in the United Kingdom.  He was also involved with the Educational Trajectory project, on the Competencies working group, and worked on the development of an e-portfolio specification called LEAP2A. Susan is from Tufts University and is the director of a system for health education called TUSK. She is also involved in a number of other MedBiquitous working groups and chairs the Curriculum Inventory Working Group.  Andria is from the Medical College of Georgia and is Associate Dean for Evaluation and oversees longitudinal data bases.  Morgan is a part of a group within the AAMC interested in using standards for educational data.  Rachel is also from England and is the Associate Dean for Student Program at University of Missouri.  She received a call from the GSA people to be involved and also is in the same office as Kimberly Hoffman who was involved in e-folio work.

Kevin is the Assistant Dean for Medical Education and Medical Technology at the University of California, San Francisco.  He worked on the Educational Trajectory project. UCSF uses Mahara, a portfolio system that leverages the LEAP2A specification Simon mentioned.  Bill is the Vice President of Academic Affairs at the American Board of Internal Medicine.  Most of his time is spent as a liaison with the graduate medical education committee, and he has worked with Bob Galbraith and Patty Hicks on milestones assessment.  Howard comes from the University of Arizona, a relatively new medical school, and his responsibility is for educational data and the new biomedical information curriculum.  He was introduced to this group through Morgan.

2.       Goals of the working group

Linda asked Valerie to provide an overview of the goals of the group using the presentation on the wiki.  Valerie reviewed that the goals of the group are to: 

  • Develop an initial technical specification for the exchange of educational achievement data across medical schools and residency programs.
  • Provide assistance to the medical education community in ensuring interoperability of educational achievement data.

Rachel asked if there was an example of what a model specification would look like?  Valerie explained that if you have access to a web browser the MedBiquitous section has a standards and specifications link.  The difference between a standard and a specification is the standard has been reviewed and balloted and the specification has been reviewed but not balloted. Valerie further explained that there are two parts to a standard: an XML schema, which contains the rules around what the data looks like, and the specification, which is a word document that describes those rules. Linda commented that the document looks complicated but Valerie handles that.  The technical jargon is not the group’s job but deciding what kind of information to include is the main task of the group. Valerie takes the business needs and puts them into a technical language. 

3.       Development process

  • ANSI
  • Schedule
  • NBME/AAMC sponsorship

Valerie explained that MedBiquitous is accredited by ANSI, the American National Standards Institute, and follows an accredited standards development procedure. The working group has an important job, but much happens before and after the group convenes. Ultimately the specification must be approved by the MedBiquitous Standards Committee to become an American National Standard.  The working group submits a draft that is voted on by the Standards Committee. It’s our goal to do that in an accelerated six month timeframe. The AAMC and NBME are sponsoring this work so that it may proceed in an accelerated fashion. The normal length of time to develop a standard is about five years.  The schedule for the working group teleconferences are from June 28 through January 3rd. 

4.       Communications

Valerie explained how this is a virtual group and communication is done through teleconferences, emails and the wiki.  She encouraged the group to visit the MedBiquitous wiki space for collaboration. There is information on the next call there and it gives an opportunity for the group to edit documents or make comments. In addition, group members can send email to the mailing list, enabling the group to continue to do work and collaborate between calls.  You will need to login in on the wiki to edit documents.  Valerie encouraged everyone to post comments or send email to the mailing list. 

5.       Project plan and timeline

Valerie continued describing the project timeline. We are beginning June 28th reviewing the use cases which is the first task for the group.  The review should be completed by July 26th, and then data analysis goes to September 12th. After that we will develop a specification and test interoperability, wrapping up January 3rd. 

Linda commented the some participants are on both the EAG and MedBiquitous groups. The EAG developed the use cases and we are going to review them. She added use cases are important to think about the kind of data we want to use.  She suggested we may want to add other groups working on what we are doing so we can collaborate when possible.  Howard asked if the wiki is set up so that it will push an email out when anything changes.  Valerie answered she usually sends out a notification via email when there is something to review on the wiki. Usually that is done via a link in the agenda for a call.  Simon asked if we are using the Educational working group space or a separate space within the MedBiquitous website.  Valerie responded that the Educational Trajectory working group has been reorganized a little bit into four main sections.  One is achievements where the working documents will go. There is a section for trajectory working documents where past educational trajectory documents reside, and a Meetings section for agendas and meeting minutes. Then there is a section on member interest started on the last project and that can be updated to include any interest or related work that anyone wants to include.

6.       Next steps

Valerie continued with a review of how the standards might be used. She asked the group if they could get educational trajectory data to move from one place to another, what would they want to do with it? The use cases actually describe how the standards will actually get used.  After reviewing use cases, examples will be collected of whatever we are trying to standardize.  Real educational achievement data is very sensitive, so we may collect fake examples of achievement records for analysis with regard to the data structure.  Bob Galbraith encouraged the group to look at the use cases.  Linda shared that the EAG group described seven ways to use data listed below:

  • Learner reviews educational achievement data
  • An external reviewer views educational achievement data at a transition point in the learner's career or education
  • Researcher compiles and examines educational achievement data looking for trends or to answer specific research questions. 
  • Administrator compiles and examines educational achievement data for graduates looking for local trends 
  • A mentor reviews educational achievement data and recommends data for presentation to an external reviewer or supports learner reflection.  The mentor can help the learner pick out data that would be useful to send out to anybody else. 
  • The learner preauthorizes an individual to serve as an external reviewer or mentor for a set of data over a specific timeframe. The learner says in advance that someone else can look at their data, during the 3-5 years of residency training, for example.
  • The learner selects a subset of data for presentation to an external reviewer or mentor 

Bill commented that he likes these seven categories and asked if there is a standard framework that informs a structure of the use cases.  Valerie explained there are many formats for use cases; MedBiquitous uses a format originally used by IBM that is simplified a bit.  An ID is used to identify the use case. Actors are people or systems that interact to achieve a goal. People are identified by their role in relation to the goal of the use case. The same person could be an internal reviewer or learner depending on how they are using the system.  There may be an event that triggers the use cases, and there may be assumptions, like the user needs to be logged into a system.  The description has a series of interactions between the users and the system. 

Susan asked if academic promotion was an example of what things could be covered.  Linda commented that use case 7 would likely support that. Alan noted that we may need to distinguish educational achievement from professional achievement. Rachel asked about the discoverability of data. Bob commented that the central e-folio connector that has been discussed would need authorization from the learner to release any information. It would not replace the current system but communicate with other systems to compile data. Once compiled, the data would be expunged from the central system. Data would not be discoverable in the e-folio connector since no data is housed in the system.

Simon mentioned that the system is not what the group is developing; rather it is the common format for the data.  Bob agreed, adding that in the e-folio connector the data stays in the primary repository and no record is kept. Patty asked about tracking of the content and what type of data was transported.  Is it business policy to not keep a record?  Bob commented that a record of what data fields were transferred could be kept. Howard asked if training ends when a person finishes their residency training.  Linda replied no; we are working under the assumption that training goes on indefinitely.  Valerie mentioned the task for next meeting is to review the use cases: note anything that should change or anything that is missing.  The next call is on July 12th.  Valerie will send the working group the link to the use cases to review.  She encouraged everyone to write down the kinds of data they want.

Decisions

Action Items

  • Valerie will send a link to the use cases to the working group participants.
  • Working group members will review the use cases and note what should change and what is missing
  • No labels