E-mail
Meeting Information
| Date: | August 4, 2010 |
| Time: | 8 PDT/11 EDT/16 BST |
Attending: Rosalyn Scott and Tim Willett, co-chairs; Valerie Smothers, staff; Susan Albright, Mary Pat Aust, Maria Esquela, Simon Grant, Ted Hanss, Logan Holt, Dave Kiger, Isarin Sathitruangsak
Agenda Items
The group welcomed Ted back. He commented that they had a competency pilot last summer, now they are scaling up. He added that managing competency and outcomes is very complicated; he is not sure how scaleable it is. Michigan has 9 competencies with 100s to 1000s of outcomes beneath. They gave the students tools for linking competencies to tools and activities. And they learned about what people don’t want to spend time doing. Use has been very inconsistent. Some students took it as a helpful planning tool, others ignored it. They now have a new assistant dean, so there may be changes in the future.
1 Review minutes of last two calls (June , May )
Tim reviewed the minutes of the last call, including the decision to add a supporting information element to the Competency Framework.
The minutes were accepted.
Tim asked Simon if he had an update with respect to contacting his European colleagues. Simon replied that he had not, but he did distribute a publication he had worked on. Simon agreed to work with Valerie as to what are the appropriate items to present to European colleagues.
Tim did review the past 6 months of discussion and created a list of outstanding issues to be discussed later on the call.
Susan was unavailable, so Dave offered to go through the Tufts competencies tool. Dave demonstrated the tool for visually displaying hierarchical competencies. They do not currently display multiple parentage, but it is built into the back end. The reason for that is that the interface allows editing, and if the competency occurs in multiple places, you need to maintain the state of the competency to build the sub trees. There are concerns they have moving forward how deep the trees may go. They also have concerns about on the fly reporting. If this is a heavily used set up, trying to build reports may be processor intensive. They are monitoring as they move forward and trying to architect well.
Simon commented there is not difference in principal between root, branch, and leaf nodes. Dave confirmed that. At the leaf level, you always have a skill or something similar. That can be managed with the type value. Something can be leaf on one subtree and a branch in another. Simon thought that was good.
They have maestro database, and they are using Pearl for programming. For this instance, there is an ajax setup.
Tim asked if in the future program or course directors would be able to log in and set up competency frameworks for their program. Then those frameworks would be available in other parts of TUSK for mapping to learning objects etc. Dave replied that was correct. Right now they are just trying to get the data entered. Isarin is working on an assessment tool. People will want to see how many lectures cover this competency.
Susan commented Isarin is linking to competence assessment for 3rd and 4th year vet school. Isarin commented that integration isn’t ready yet. Susan added that they are also building a question bank and want to link questions to competencies.
Tim asked how the database tracks order. Dave replied that it stores the current position in tree. If you have A and B, and both have child C, it stores those child locations independently. Whether or not order means anything, it is a nice tool for managing the competencies.
Simon asked if types had clear definitions. Dave replied no. Every school in their environment (vet, dental, etc) may end up with their own types. They give people the ability to maintain their own types. One of the early specs called out a specific list of types. They used that as a foundation initially, but it didn’t make sense for them. It bred confusion because so many types were not used.
Valerie asked if there were any disconnects with the current specification. Dave replied no. The only concern he had initially was that he wasn’t sure how they were representing root competencies. That has been clarified since. Now it is pretty solid. He may find out more with increased use.
Tim commented that currently the their system only supports internal competencies. Tim asked whther that reflects the fact that users do not need to incorporate competencies from other organizations. Dave replied that was correct. They have discussed the potential need to import and export from other formats.
Susan asked how would they show relationships to external competencies? Dave replied that he was not sure. For their purposes, it has to exist in their system.
Isarin added that there could be more database tables and relationships, extending what is currently there.
Dave added that at one point, Scottish doctor posted competencies in a particular format. They could have an external reference to that. He questioned how reliable that information would be. If they change the definition, is there a system to make sure everyone knows about the change?
Valerie asked if regulatory questions had come up, such as mapping competencies to specific LCME or veterinary accreditation requirements. Susan replied it had not come up. They were asked to build a system for competence assessment. They had to convince the schools that it made sense to post competencies. She will ask them and see. They are starting accreditation this winter.
Dave commented that they have talked about LCME keywords, and having accreditation body keywords as their own thing. It may make more sense to have it wrapped into this.
Tim commented that it’s been a conceptual approach up to now. If we have two distinct frameworks, we would provide subsequent spec to map one to another. That way you don’t run into confusion about what is yours and what is theirs. There would still be broader than, narrower than, equivalent to, etc. Logically, you ought to be able to trace paths. The need for the subsequent spec is becoming more apparent.
Susan added that she looks forward to speaking more about it with this group.
Tim asked if users could export an XML version of the framework conformant with the spec using the Tufts tool. Dave replied that was a planned addition.
Tim walked through several of the outstanding issues. We are using LOM as container for metadata about a competency object. He commented that it is a good idea to explicitly state which fields from LOM we would require or recommend. He reviewed and made a list of elements he thought were relevant. Identifier and title are both required.
Simon commented that we should make several things equal between objects and framework.
Valerie clarified that description is recommended in the current Competency Object specification.
Rosalyn agreed that was a good idea. Tim commented that some frameworks just have competency statement themselves. Rosalyn recommended having something a little stronger than optional. Simon asked about formatting. Some have no description, others have a long description with formatting and links. That could be limitation later. Tim offered that we could add the supporting information element to the competency object. The group agreed.
Tim then discussed lifecycle version. That element could be critical for a competency framework. If a given framework refers to an external competency, and that competency changes, the mapping may no longer be relevant. For comp obj, he recommended not allowing versions. If there is a meaningful change, a new one should be published. If need be, you can issue a new version of the framework that incorporates the new competency. In general, competency frameworks are updated from time to time, and a version would be required.
Rosalyn asked what would happen over time. You could end up with kludgy kind of framework. Valerie commented that many organizations come out with a new version of their framework every year or two. Tim added that CanMeds hasn’t issued a new framework in 6 years.
Maria commented that in nursing and pharmacy, there are quality of care indicators that are being narrowed as they get more data. If you wanted to use quality indicators, you wouldn’t want to delete past versions, even if not current, because data may be mapped to that. You may retire something, but you don’t delete it.
Simon added that in the web standards development community world, the URL redirects to the most recent version of something. Or you can provide a URL to go to a dated version of a specification. He sees the same argument applying to competency frameworks.
Tim asked if you publish a competency object, and you have version 2 of that object, would it overwrite or become a new competency object? Valerie commented that is a matter of best practice. The group can provide some guidelines on that in their best practices document. Tim asked Valerie to examine how lifecycle is dealt with in more established standards like LOM, and we will present that on the next call.
4 Review revised competency framework specification
5 Open discussion
Decisions
Action Items
- Simon will distribute the most recent specification to his European colleagues
- Valerie will add Supporting Information to the Competency Object specification.
- Valerie will investigate how lifecycle versions are dealt with in LOM and other standards and present on the next call.