December 6, 2017
11 AM PST/12 PM MST/1 PM CST/2 PM EST/7 PM GMT
Attending: Ellen Meiselman, David Topps, Co-Chairs; Erick Emde, Jeff Korab, Valerie Smothers, Adam Tarnowsky, and Charles Touron.
1 Review minutes
The minutes were accepted as submitted.
2 Recap of what the U of M Nursing school is doing with respect to mannekins and Caliper
Ellen stated she attended a meeting and started planning for using Caliper to track mannequin simulation data at the nursing school. She showed them the College of Chest Physicians with very rich verbs, and they chose to put meat of things into the verb. Their thinking is if they attack it from that angle and intercept from the EMS system, they may be able to add Caliper at that point.
Valerie questioned what factors went into choosing Caliper and if they considered xAPI. Ellen commented that the Caliper people were looking to engage communities of practice and approached them.
Jeff mentioned their approach is similar to the nursing school. They are getting access to simulator data and log files and doing post processing. They are working to expose that data. Adam questioned what LRS they are using. Jeff answered Learning Locker. Ellen asked if Learning Locker has existing visualizations. Jeff noted they use it as statement repository to gather data.
Valerie cautioned if Michigan’s work is used to develop a standardized approach, it is in direct competition with what we develop. Ellen added Caliper is in direct competition with XAPI. Many folks trying to map between them. Valerie noted that we had many developers using IMS QTI with the MedBiquitous Virtual Patient standard. We were not able to directly reference QTI, but instead provided some guidance on how to integrate QTI data into a Virtual Patient. We could not build directly on top of it due to licensing issues.
Valerie suggested creating a standard set of verbs for mannequin XAPI. David noted the profile list of verbs is complete, but we cannot find a vendor interested in implementing. Valerie noted the Chest Physician verbs were more clinically specific. Ellen suggested having an offline discussion about that.
3 Charles Touron Introduction
- Announcement on TCM ATIS with diagram
- DOD INSTRUCTION 1322.26 DISTRIBUTED LEARNING (DL) (DODI)
- IEEE/ADL Actionable Data Book , also see IEEE minutes
- IMS Extended Transcript
Ellen mentioned that Charles works for the Army Training Information System defining requirements for new systems. He is currently working on a white paper for IEEE. He is working with ADL and looking at specifications for learning in general. He is the ADL lead on all Department of Defense standards and specifications. He would like to learn more about our activities. He is funded to replace active training systems and deal with lots of training needs with limited customization but many products.
Valerie asked about standards related to healthcare professionals. He noted he was asked by aviation for some hands on training material with aviation mechanic using real engine without breaking it. They use live participants, augmented reality and other sensors.
Adam Tarnowsky is a Product Manager from One145, doing scheduling, assessment and tracking for postgraduate medical education and continuing medical education in Canada. He is working to open up their system and talk to other systems.
4 Discussion on tracking behaviors and allowing competencies or best-practices for learners to emerge from the data.
Behavior patterns may emerge from the data that will inform our understanding of the thought processes or behaviors of a high- or low-performer.
In online apps, behaviors can be tracked as clicks or nodes, but in real-life situations, like clinical encounters, it may not be so easy. Question: In the real life medical education or medical activity of your choice how would you track significant behaviors? Not all of them would be part of the activity itself. Is this something that is already tracked in some manner? Is there anything you wish you could see that would help in assessment, that isn't currently tracked?
Ellen noted that they intended to have a discussion on how to best approach competencies within various activities. David had extensive discussions on the differences between top down and bottom up design. He is more of a bottom up devotee but he respects and promotes the top down approach.
David noted that he did not know what questions researchers would be working with, so he used a bottom up approach. This was partly based on Open Labyrinth having a lot of back-end tracking. Things we thought were useful were not at all and vice versa. How the actual performance compared to peers was particularly powerful. Independent assessment has alleviated past observer biases.
Ellen asked if this would be appropriate for K-12 education, as she is currently involved in an intelligent student centered dashboard. The goal is to show level of effort in some incorruptible way using xAPI. Valerie questioned whether the work was connected to work of Johmarx Patton. Ellen mentioned it was not even a prototype yet.
Valerie commented it sounded like two different use cases, one using emergent properties to identify learners that you are concerned about and also use case for more specific assessment. David added they are looking at how the scenario performs, not just the learner.
Ellen asked about CME work using Open Labyrinth for assessments. David confirmed they are setting up virtual workplaces and EMR’s. Ellen is interested in xAPI profiles for patient encounters, EMR work, collecting data on everything done in EMR. David noted there are confidentiality issues with identifiable patient data that becomes a security problem. He is looking to add an xAPI interface onto existing software to track what they do rather than the content of what they write. The actions tell the story.
Valerie commented the Call for Abstracts is now open. The conference is set for May 20-22 with Orientation on Sunday followed by full sessions on Monday and Tuesday. The themes are similar to previous years. Abstracts can be submitted through January 5th. The presentation types are similar with one change. The Innovation Demonstrations will have a period afterward to talk with presenters one on one. They will be grouped together for informal discussions. She announced David Cook, from Mayo, will provide the Keynote and Teodor Grantcharov, the developer of the surgical black box, will be providing the closing plenary.
Jody will set up a time to discuss mannequin profiles with David, Ellen, Erick, Jeff and Valerie.
6 Discussion of a mannekin profile (group) – Profile: 2. Human Patient Simulators, Mannequins & Task trainers
GDrive folder as workspace? – https://drive.google.com/drive/u/0/folders/0B2gSucgM9Y3NQ0U4bFRTT2lVaTQ