July 18, 2007
11:00 AM EDT
Attending: Valerie Smothers, Vladmir Goodkovsky, Tim Willett, Doris Quinn, Ted Hanss, David Stern, Tamara Stein, Mary Pat Aust
1 Introduction of new participants
Valerie began call by introducing Ted Hanss from University of Michigan and Vladimir Goodkofsky from University of Virginia. She then asked David Stern and Ted to describe a competency project that they were working on. David explained that Michigan is interested in developing new learning experiences tied to outcomes in education, an outcome based undergrad curriculum. To generate outcomes, best way is to sort by the way physicians experience patients in clinic. They use the Canadian presentation of complaints as a way to organize the curriculum. They have taken 120 conditions and mapped them to differential diagnoses and then have identified through use of external resources outcomes associated with each compliant. For diarrhea, there is a differential list and set of outcomes related to diarrhea in nine domains (patient skill, professional behavior, diagnostic reasonsing, therapeutics, etc.). Ted then explained the LMS project that they are working on. The vision is to provide a mechanism for tracking that is learner focused as opposed to course focused. The learner tracks their own progress using the LMS. He is investigating a number of tools now, including Virginia's adaptation of g Moodle. Indexing is the challenge. They have a social bookmarking pilot, to see if students can index content.
Doris asked if they were familiar with the Vanderbilt knowledge map. They have mapped out med school curriculum. If she wants to look up diagnosis, the map will tell her where it is taught. Anderson Spickard is the physician lead. Tim commented that everything is indexed to MeSh. Doris added they are eager to share.
Valerie then asked Vladmir to introduce himself and tell the group about the ACTS project Vladmir explained that his PhD is related to adaptive instruction for diagnosis. He has specialized in adaptive instruction. He focuses on systematic design and developing reusable tools, including frameworks and instructional adaptive engines. He has participated in the IEEE's Learning Technology Standards Committee working group on learning object metadata. He is working on the Achieving Competency Today project, which is intended to provide educators with tools to review and fill gaps in student competence. They have a system that implements it, and they are testing the tool. Educators can describe competency, available learning activities and assessments, and connect them together. The tool allows them to organize the curriculum to reveal and fill gaps in the curriculum. He has tried to summarize what he knows about competency in a document that he will share with the group. It's possible to name 4 systems: patient, doctor, learner and instructor model. He has 4 or 5 definitions of competencies based on these different kinds of frameworks.
2 Review minutes of last meeting
Valerie asked if the group had any comments on the minutes of the last meeting. Doris commented that Eduardo Salas is a hard catch. Valerie suggested reporting back to the group. The minutes were approved.
3 Review consolidated use cases
Valerie explained that two previous use case documents have been consolidated into one document. The new document is more abstract and has captured the information in a way that is relevant to many education and assessment environments. Use cases are intended to capture a person's goal in using a system. The bottom of each use case, potential standards are identified. Competency framework is key to almost all of the use cases. The first use case is Associate curriculum components with competencies. Valerie provided an overview and asked what data structures working group participants used for their competency frameworks.
Doris commented that she is not too familiar with their curriculum tool. She will have to get back to us. Vladmir commented that his work is domain independent. They use a hierarchical tree of competencies where children can have more than one parent- a directed acyclic graph. It's more difficult to role up assessments. Tim commented that his experience hierarchy and polyhierarchy is important, interconnecting frameworks is important. As far as elements within the data, outcomes structures are just text documents. For his project, they included a unique identifier, title, description, comments, synonym, and fields that describe the relationship. Aside from hierarchy, it's also important to have a similar to or related to key. A competency may be related to a competency within the same framework or one within a different framework.
Valerie then provided an overview of its case two, Search a system to develop a report that details what competencies are addressed in the curriculum. This use case is intended to reveal gaps curriculum. Tim asked if in addition to the users selecting the competencies and seeing what parts of the curriculum address those competencies users could also select parts of the curriculum and see what competencies address those. Tim added that the University of British Columbia they captured not just where but to what degree a competency is being addressed. They used Bloom's taxonomy to indicate how is being addressed. Ottawa handles it differently, indicating a weak or strong association with a competency. Say an educator is reviewing an MI resource and wants to decide which topics are addressed. MI would be a strong association; if there is some content on risk factors, that may have weak association. David agreed that indication of the strength of an association would be useful.
The group then moved to the third use case, Use a machine readable set of competencies or outcomes developed by a collaboration or another organization (import it into the system). Tim explained that Rachel Ellaway manually input the Scottish Doctor learning outcomes into vdex to import it into her LMS. That's the only instance he knows of importing a set of competencies. They've been trying to get TIME into CURRMIT and other LMSes. There is definitely a need. Tim added one challenge they've encountered is updates. Is there a way to get updates reflected in the LMS? How can you avoid screwing up mapping? Valerie suggested authoring another use case, and Tim offered to help with that.
Mary Pat added that AACN develops competencies for job profiles; they could define competencies and push them out to facilities using their competencies. There would be a benefit to users and hospitals. The would also be advantages for certification, how do we determine competence and facilitate that.
Tim asked how working group members should act on their ideas for expanding the use cases. Valerie recommended that they send an e-mail to the group so that they could coordinate their efforts.
4 Discuss needed components of a competency framework
- Tim and Valerie will add a use case on updating a competency framework within an LMS
- Group member will send any ideas for modifications or additions to the consolidated use cases as an email to the group.