August 29, 2013
8 PDT/9 MDT/10 CDT/11 EDT/16 BST
Call in Number
AUSTRALIA BRISBANE: 61-7-3102-0973
Attending: Tim Willett, Co-Chair; Connie Bowe, Terri Cameron, Kelly Caverzagie, Mary Jo Clark, Stephen Clyman, Robert Englander, Linda Gwinn, Kevin Krane, Deborah Larimer, Karen Macauley, Dan Nelson, Paul Schilling, and Valerie Smothers.
- Review minutes of last meeting
Tim summarized the minutes from the last meeting. The bulk of the discussion was centered around identifying thresholds. The group came to a consensus on how to include the threshold element and how threshold is met. The group also discussed approaches on grouping levels. Valerie noted that in the current specification either approach illustrated (using labels to indicate groupings or expressing a single level as a score range) is a viable option. Mary Jo commented on further distinguishing scores within a range and indicated that behavioral examples for different scores would be useful. There were comments about referencing competencies and implementing at Tufts and Tulane. Valerie was going to follow-up with Steve Lieberman. Mary Jo moved to accept the minutes as submitted and Bob seconded the motion.
2. Discuss resolution of competency references from Technical Steering Committee
Valerie had a discussion with Joel Farrell, chair of the Technical Steering Committee, about the problems leveraging Dublin Core Metadata. Together they decided to cease using Dublin Core for competency references and instead create an element in the MedBiquitous namespace. There are Resource Description Framework (RDF) elements and attributes that point to a competency and competency framework. This approach uses the existing RDF standard and is in line with the principles of linked data. Joel and Valerie presented this to the full Technical Steering Committee yesterday, and they agreed with the approach. Valerie distributed the revised specifications that reflect those decisions with the agenda. Tim noted this will be discussed again in item four on the agenda. Valerie added that from the perspective of the educator it makes no difference; from a technology perspective, it’s a small change.
3. Discuss results of developer and educator reviews (Tufts and Tulane)
Kevin did not have much to report because their developers are still looking at the document. Valerie commented Susan was not on the call due to attending the AMEE meeting. Deborah didn’t have anything else to add. Valerie encouraged Deborah to bring any items back to the group for discussion if she finds anything. Tim asked if Susan’s developer was on the call but he was not so feedback on this will be deferred until next week.
Valerie began walking through the power point which incorporated feedback from the last call. On slide six there is a description of a threshold. Previously we just had score; it’s now minimum acceptable score. Tim noted this implies anyone with that score or higher has achieved the threshold. Tim suggested including Threshold in the summary diagram on threshold on slide two. Valerie agreed.
Valerie continued with a discussion on what Steve Lieberman’s group at the University of Texas is doing. Their initiative is TIME, Transformation in Medical Education. It’s a University of Texas initiative to use a competency-based approach to premedical and medical education making the medical school experience shorter. Valerie had an opportunity to talk with Steve yesterday and he was pleased with the specification’s ability to capture the TIME requirements for certain milestones to be met prior to transitioning to the medical campus. Valerie included a link to the PDF from the competency agenda and it’s also available on the wiki. Steve had a question about authors and wanted to know if you can have an organization as the author. Valerie answered it was absolutely OK to do that and she mentioned you could also list individuals or members of a committee; either or both would be fine. Slide fifty one includes the definition of the performance scale. TIME had seven different levelsI, II, II+, III, IV, V and VI. So II+ actually has a score and position of 3, but the label is II+. Slide fifty-two describes one of the rows in the TIME framework, the title, and how it relates to a competency in a competency framework.
Slide 53 shows how you would establish the threshold for transition to medical campus for a specific transition milestone. The minimum acceptable score is 4 which translate into III. At the time of transition from the undergraduate to the medical campus, the student will have demonstrated the ability to demonstrate compassion and respect for a patient in a stressful encounter when faced with a purpose?designed standardized patient based simulation challenge at the following level.
Steve was happy with the way things were expressed in the TIME framework; however, he did have a reservations about using the word score given their interest in competency-based grading. But he added he was willing to look past that given how interwoven the notion of score is in the specification.
Tim commented that for TIME, the actual performance framework for them is a collection of components that indicate the use of the same common 1-7 scale, and the threshold associated with each transition milestone. He asked if there was a developmental progression to the milestones; Valerie replied she did not think so.
Valerie began walking through the specification. The diagram on page twelve shows the structure of the framework at a high level. Solid lines indicate an element is required; dotted lines indicate the element is optional. LOM allows one to include descriptive information about the performance framework including title, author, publisher, etc. LOM and a few subelements are required. Effective date and retired date are optional. The date format is specified by XML. The specification supports versioning. Replaces lets one indicate that this performance framework is replacing an older version. You can also indicate that this performance framework has been replaced by another performance framework, a newer version.
Valerie continued to page seventeen where there was a high level view of Healthcare LOM. She noted if you scroll down you can see we are requiring title and identifier. Some required elements are containers used for organizational purposes. The identifier entry must be in the form of a URI. The best practice is to have the URI used for the identifier provide information about the performance framework, such as the XML document. But that is a best practice, not a requirement. Several fields are recommended, description, status, publisher, educational context, copyright information, and target profession.
Valerie continued to page twenty five. Supporting information field references or includes supporting information for the performance framework as a whole. You can referencing out to an external document using the new MedBiquitous reference mechanism or include support documentation using XTML.
Page 26 provides more detail on the reference element; Valerie will add an image before the next call. Valerie explained that the schema allows us to say that two elements from the RDF namespace are required. The specification tells you which ones: Description and type. Description uses the RDF about attribute to indicate the thing that you are describing. In our case, that is usually the URI of a competency or competency framework. Type uses the RDF resource attribute to indicate what kind of resource we are referring to. For competency frameworks, this will be the competency framework namespace, which indicates that the thing being referred to a a competency framework conformant with the MedBiquitous standard. For a competency object, the resource attribute will be the namespace of the competency object specification. Description and type element are intended to work together, indicating what I am describing and then what type of resource it is. There are two references when pointing to competency: one for the competency object, one for the competency framework. If you are pointing to a PDF document, you could use the Dublin Core format element to indicate you are pointing to a PDF document; that is optional.
Tim asked if mashing up RDF and XML is an OK practice. Valerie answered yes, it was. RDFXML allows RDF elements to be integrated in XML data. This is going to be useful for leverage tools that are out there. Tim asked if RDF triples are always represented. Valerie answered not in this case they are not represented. Triples are the ability to specialize a relationship. Tim read this book, I did this. We have a simpler use of RDF, saying this resource is being referenced and stating the type of resource. If anyone has questions after the call, email Valerie directly.
Valerie continued on page twenty-seven. Performance scale defines a scale used within a performance framework. There may be more than one. For example. in Internal Medicine, one scale is used for assessing detailed competency but then a different scale was used for assessing the competency domain as a whole. Each performance scale has an id so that is may be referenced from components in the framework. The PerformanceScale element lets you define your scale, ie 1 to 7, and which score is least competent and which is most competent.
Page twenty-nine provides an overview of the Component element, which lets you express the levels of performance relevant to a specific competency. First there is an ID for the component. Title is mandatory. There may be an optional abbreviation. The component may reference more than one competency if the same set of performance levels is used to assess multiple competencies. You may also indicate the author and reviewer of the component. Tim asked if it was OK to have a string for author and string for viewer rather than leveraging LOM. Valerie shared this is a lot simpler and OK to do. The next thing is thresholds and then additional information, described on page thirty-one. Finally there is either a component reference or performance level set. The component reference ddresses some of the more complex examples in pediatrics where a matrix of performance levels existed to assess a single competency. For all other performance levels that are not expressed as a matrix, you would have a performance level set.
The competency element on page on thirty-three at the bottom uses the reference element that we already discussed. We will provide examples and flesh that out a little more.
Thresholds are described on page thirty-five. There is a title, optional description, and then minimum acceptable score.
On page thirty-six there is additional information. There are two attributes: label and position. The label attribute is required, and we provide the following recommended values: assessment method, background, example, note, reference, and resource. That approach allows for flexibility. After that there is position, so that notes can be ordered. Additional information has three elements. The first is Text, which is not formatted but is required. In some cases, there may be a reference to an external resource. The second element, Reference, lets you point to the URL. That can be useful for pointing to educational or assessment resources. Lastly XHTML formatted text may be included using the div element.
Performance level set is described on page thirty-nine. The performance level set indicates how we are assessing performance for that competency. There are two sub elements, scale and performance level. The scale indicates if you are assessing on a scale of 1 to 10, to to 5, etc. Then each level of performance is defined.
Performance level is described on page forty-one. The first thing is display order, which is required. That indicates if this performance level should be displayed first, second, etc. Then there is a score associated with this performance level. That score can be single score (2) or score range (1-3). There is an optional label for the performance level (for example, critical deficiencies). There can be one or more indicators (also described as behavioral markers) associated with this level of performance. Finally, there is that additional information to point to resources and provide footnotes regarding this performance level.
Tim asked why indicator is required but label is not. Valerie answered in many frameworks not every level has a label. Tim commented we should stop here and continue next time. Tim added hopefully next time Susan will be with us. Tim asked Valerie to send out a reminder to Susan and Kevin about the upcoming September 13th meeting and for them to call Valerie with concerns ahead of time. We will start with page forty-one next time.
5. Open discussion
- Valerie will add information about the Threshold element to the diagram on slide 2 of the illustrative powerpoint.
- Group members will email Valerie any questions about the specification.
- Valerie will send a reminder to Kevin and Susan regarding the Sep 13 call.
- Valerie will update the examples and diagrams in the specification where necessary.