February 26, 2010
8 PST/11 EST/16 GMT
Attending: Kim Hoffman, Chair; Valerie Smothers, Staff; Brownie Anderson, Elaine Dannefer, Maria Esquela, Maureen Garrity, Simon Grant, Bonnie Helevic, Lindsey Henson, Chandler Mayfield, Amber Montañano, Morgan Passiment, Kevin Souza, Sandra Waters
1 Review minutes of last meeting
The minutes were approved.
Elaine explained that the extracurricular activities small group revisited the issue of publications. The larger group proposed links alongside the paragraph describing the program. The links would point to outputs of the program, such as publications. The group recommended encouraging the learner to include the attachment or link to a publicly accessible version. With regards to verification of enrichment activities, they rethought suggestion for a reference and instead proposed that students attach an actual work product or testimonial from a supervisor supporting their activities.
Linda Lewin had provided an additional comment that Valerie described. Because we have very broad categories such as enrichment, there's a loss of richness in the visual display. NIH fellowships and HIV service activities both show as enrichment activities. Simon had suggested providing a mechanism to highlight the top three activities. That would relieve the group from further classifying enrichment activities. Highlighted activities could be displayed with gradation of color. Carol had commented that would be useful to residency program directors.
Kim asked for further clarification. Would the line be a different color? Simon replied that there would be some visual cue. It could be different color or a brighter color.
Kevin commented that it was an interesting idea but questioned how it would play itself out. He questioned how educational trajectory data would be archived, and whether the learner could change their top three activities after graduation.
Lindsay added that one reason schools can send the medical student performance evaluation letter is that it is a fixed document. She thought the educational trajectory document should remain fixed as well. She added that she liked the idea of students figuring out their most important enrichment activities.
Valerie commented that when looking at the structure of educational trajectory data, the group should stay focused on the use case of the learner transitioning to residency. That is the transition the data is intended to support. What happens after that point is less important.
Kim asked Kevin if he had any further feedback. Kevin replied that his group had begun to pull data together, and he found it a little overwhelming. From his perspective, being able to see the top three activities is essential. Their data system tracks many different extracurricular activities. The learner doesn't want to send it all: that is too much noise. Having a mechanism for the learner to prioritize this data is a good suggestion. If we have just-in-time data transfer throughout the learner's career, they could tweak what is highlighted according to their needs.
Kim replied that she did not see that as an either or. We could have that and also have a static picture in time.
Maria commented that she had worked on an MSPE where the learner was unsure of their plans after graduation. What activities are important may depend on which type of program they were applying to; they may want to have two different profiles.
Maureen commented that one directive they have for MSPE is that it is not supposed to be career specific. It is an evaluation of student performance that is deliberately generic.
Kim summarized that the group agreed in concept with what the extracurricular group had put forward, highlighting a specified number of contributions.
Valerie provided an overview of the technical architecture and then showed how the concepts from Atom and leap2A were represented in the John Doe trajectory diagram. The trajectory as a whole has a title, author, date and identifier, which takes the form of a URL. Each entry shows an activity within the trajectory, which has a title, start and end dates, date the entry was updated, and identifier. In some cases there is a description; for the case of coursework, there is a link to the program description.
Simon commented that just because the identifier is a URL does not mean that anyone can access the trajectory. You could still have access controls over the data.
Lindsey commented that the way the formal curriculum was shown it looks like more time was spent on the curriculum in certain years. Valerie agreed to work on the visual to correct that misrepresentation.
Lindsay commented that the link to the description of the program was likely to break over the years. Valerie replied that he could be included in the trajectory instead. Simon commented that we could have the link go to an attachment that is included within the educational trajectory document.
Lindsay added that we may want to mandate author for an entry. Kim agreed that specifying the author should be consistent across all entries. This removes any confusion as to who is attesting to the activity.
4 Discuss UCSF lessons learned
Kevin introduced Bonnie , who is a cornerstone of UCSF operations. Kevin and Chandler reviewed the spreadsheet that outlined the data fields. They went through to determine there are 29 data fields. They found where each one lives. For 29 fields, they identified 5 potential sources:
- School of medicine student information system
- Registrar system
- External institutions
- Self reported data
The school of medicine information system tracks information about assessments, extracurricular activities, fellowships, scholarly projects, etc. The registrar system tracks leaves of absence, dismissal, but not details of why those events occurred.
Their e-portfolio uses leap2a. That is where much enrichment activity data lives, including competency related data. Some also lives in student information systems. Some live at another intuition if the student does a secondary degree (MPH at Harvard), in which case data comes from another institution or is self reported.
So far they have spent 4 hours trying to figure out where data lives and how to pull it out. 2 sources are non-electronic. 3 people have been involved in compiling data. 8 hours were spent querying systems and designing the xml.
Bonnie pulled together the xml feed. Her task was to create an xml schema to describe the data. It was a good way to import data because it eliminated variables. She made a schema that is 70% complete. Filling with data is a tricky step. Highlighting implies that there will be a repository where student can access the data. If they are compiling on the fly, it is challenging. They went through data sources manually identifying which pieces were relevant. Having a repository would make a lot more sense.
They selected 5 sample students. Bonnie felt as if she was replicating a lot of data; for example, a leave of absence and a Howard Hughes Fellowship may be concurrent and related. Also, it wasn't clear how to describe joint programs. Also students slow down their time for medical reasons, but that is not really a leave of absence.
Kim commented that was an interesting point. Students may remain enrolled but take longer to complete. Bonnie replied that that might be obvious visually.
Kevin explains that they decided to take five students: one MSPE , one JMP, one regular M.D. in four years, one regular M.D. in five years. The MSPE took graduate work that took 10 years.
Kim commented that we were running out of time and should start with this on the next call. Kevin agreed to distribute documentation with further information.
Kim encouraged to the group to e-mail her with other items for the agenda.
5 Open discussion
- We will include a list of publications along with links to publications included in the entry for the relevant activity.
- Instead of references to verify enrichment activities, students may attach work products or include a testimonial from a supervisor.
- We will include a mechanism to highlight a set number of activities that the student deems most important.
- Program descriptions will be included in the educational trajectory, potentially as a separate entry that is linked.
- Valerie will implement changes to the specification and diagram to reflect the group's decisions
- Valerie will work on John Doe visual to correct representation of formal curriculum
- Valerie will find the mute code and sent to the group
- Kevin will distribute documentation of UCSF data compilation work