January 15, 2010
8 PST/11 EST/16 GMT
Attending: Kim Hoffman, Chair; Valerie Smothers, staff; Michael Barbouche, Carol Carraccio, Elaine Dannefer, Bob Galbraith, Gwen Garrison, Maureen Garrity, Simon Grant, Alex Haig, Lindsey Henson, Linda Lewin, Chandler Mayfield, Pat O Sullivan, and Sandra Waters.
1 Review minutes of last meeting
The minutes were approved.
2 Agreed upon principles for pilot project
Kim explained that this document stemmed from a discussion within lasts week's EII conference call. It provides greater clarification for how the pilot will unfold. This provides clarity for how we will be sharing data. For the MedBiquitous group, it is a point of information.
3 Small group reports
Linda and Elaine talked yesterday. Linda relayed that in the small group call in December, Elaine proposed a model for extracurricular learning activities and the group endorsed it. The biggest concern is the purpose for collecting the data. You can split data until it is very complex. They felt that curriculum data could be divided into learning activities towards MD degree, and learning activities not towards the MD degree. There would be a color for pursuing formal curriculum, which correlates to activities on a transcript. Everything else would be learning activities not towards the primary degree. Within this there are two large groups: enrichment and learning activities towards a different degree or certificate program. Lerner college of medicine does have a research certificate. Other learning activities are professional enrichment activities. Then question is do you need to divide enrichment into different categories? It becomes confusing. For a pictorial representation, you only need to show that it was professional enrichment. The name of the activity would come up with further description. Linda asked the group if they felt more categories were needed.
Lindsey commented that it made a great deal of sense and simplified things. They have similar situation with certificates. Simon commented that it made sense. It is important not to introduce distinctions which are unclear. Pat commented that if we keep our mind on the portfolio, people can link in more information. The approach Linda outlined gives guidance without over specification.
Linda commented that they did not get into verification. If things are towards a degree, there will be a transcript. She is happy to go with group's approach towards verification.
Kim asked if the group considered whether text should be codified? Linda commented that as soon as you start making codes, the learner has to choose one. For that reason they chose free text descriptions. For research it may be nice to have codes, but for the purpose of someone looking at trajectory, it's good enough to have free text.
Chandler asked about what consitututes curriculum. If demonstrating competency is part of curriculum, does anything demonstrating professionalism make it? Or just what was on the transcript? Linda replied we were talking about what student had done logistically. That's another layer, how it relates to competencies. She asked Elaine how they handle it. Elaine commented their students can use any learning experience to document competencies.
Kim replied that in previous discussions evidence of competence was identified as a second tier approach. Kim commented that she was thinking about formal curriculum as those things on the LCME report leading to the MD degree. Linda agreed. The degree related things would be recorded on transcript. Kim agreed that the medical school determines what is part of the formal curriculum.
Kim commented that at the small group session at the AAMC meeting in November there was a robust theme around need to verify the accuracy of informal learning. She asked those who were there how important that is from a residency perspective. Carol commented that they don't verify informal learning now. Linda commented that while that would be ideal, there would always be things not verified. People are concerned that learner is exaggerating. No matter what system you set up, you won't necessarily get rid of the problem. If you are hiring, you may need to ask for additional documentation. Carol added that a director could always ask for verification.
Simon commented that the approach many people take is to allow learner to state the evidence. Particularly with extracurricular activities, it is difficult to imagine uniform verification. If we allow linked record in trajectory where people can say how to verify, that would accomplish a great deal.
Linda agreed. Looking at the application, you just want to know what activities the student was involved in. But if a phone number is there, the learner will pause if they think it will be checked.
Valerie commented that exploring certificate and joint programs further would be helpful to her in operationalizing the group's comments and developing the specification/guideline. There are programs where some course count towards both he primary and a secondary degree or certificate.
Lindsey commented that they have a Flexible MD program. Students take up to a year to do additional learning experiences. The experiences don't count toward the MD degree, but the activity is approved by the school. They do a lot to go out of their way to describe so that it explains the extra time to graduate. Students have to turn in a report at the end of the experience. In most cases it's not as formal as certification. In some cases the student may get an MPH.
Pat agreed that approach would work for UCSF pathways to discovery. It's not a certification. The student can go on for a degree in public health. You will have credits on your transcript. This needs narrative in the current letter. Kim commented that it would work well for some of the programs in Indiana.
Linda asked for those that have these kinds of program, if a brief description of them would be helpful, to see how it would look in the current schema. Kim agreed it would be helpful. Lindsey agreed. Linda asked for a student specific narrative description. Describe what they did over time. Then we can figure out how we would show that. Lindsey and others will send to Valerie by Jan 22.
Technical Architecture (Pat's data, LEAP2A representation)
Chandler summarized that they technical architecture group has looked at how Leap 2A could work for us. Chandler summarized that Leap 2A is a specification for e-portfolios. It builds on the Atom standard, which is used for blogs and news feeds. It's a flexible, generic specification. It doesn't have a lot of the specificity we are looking for. We've been wrestling with how to handle the specificity. The more we can keep it generic and flexible, the better. Simon and Valerie took Pat's data and translated it to Leap 2a. There's an XML file you can download and look at. At UCSF they have an e-portfolio that can import and export Leap2A called Mahara. They've been able to do successful import and export in Mahara of Pat's data in Mahara. But Mahara's trajectory doesn't have a lot of structure to it. They are waiting for categories to get hammered down a bit more, then we can see If Leap 2A needs to be extended or be used as is. With Simon's help we've understood more about Leap 2A. UCSF has also discussed with the Mahara developer. We will have to move forward with imperfect information to make next steps. Also we need to think about how those with no tools would they get their information into this type of specification. There will be lots of diff types of data.
Simon asked if UCSF were planning to write a medbiq module to add to mahara. That would allow learners to put in and view Educational Trajectory categories. Chandler commented they haven't gotten that far. Even within the institution they could use such a spec. They've discussed using the spec to move data from the portfolio and the student record system. So it has already been helpful internally in thinking about the question "What should tools do in our ecosystem?"
Kim thanked Chandler for the report. She is glad the group is thinking about how all medical schools will be able to move data around. UCSF is more technically sophisticated than others. The rest range from home grown systems to pen and paper. How do we operationalize in this environment? She asked if they would consider that as the next level of questioning.
Chandler agreed to consider the process of operationalizing. He asked Valerie if she had some thoughts. Valerie commented that implementing a spreadsheet version of the specification was one important step, and that Michael may have other thoughts given his background in extracting data from varied sources. Gwen added that the AAMC has produced MCAS and ERAS and has resources describing the technical capacity needed to implement a client system. She agreed to compile some information for a future call.
Kim asked the schools the degree of assistance that would be needed. Linda replied that she did not know. Lindsey replied that some data would be easy to obtain, but some of the data is still on paper. When she writes a deans latter, she has to go through a paper file on the student and interview the student. Maureen agreed.
Michael commented that determining what data matters is an important step. Helping school learn to align their data will have many benefits. We should reach for the lowest common denominator to allow the broadest participation.
Kim asked Chandler and Valerie to try and schedule a small group call before the next working group call.
4 Schedule of future calls
The dates for future calls are available on the wiki.
5 Open discussion