April 21, 2016
9 MDT/10 CDT/11 EDT/16 BST/17 CEST
Friday, 1 AM AEST
Attending: Ellen Meiselman, David Topps Co-Chairs; Tom Creighton, Erick Emde, Jason Haag, Olivier Petinaux, Valerie Smothers, Radu Vestemean, and Luke Woodham
1 Review Minutes
Minutes were approved as submitted. Ellen introduced Olivier Petinaux, a distance e-learning manager from the College of Surgeons as the newest member of the group.
2 Development update
David provided an update on Open Labyrinth. They are sending xAPI statements both in real time and post hoc so that they can look at data for cases completed a few years ago. He is writing to five different LRS’s about xAPI grammatical differences in the statements. Some are more flexible in terms of what they will accept; others are very strict. Tom noted that ADL is building an LRS validation tool. David added that they are working with HP5 widgets in Open Labyrinth to capture activity for mixed scenarios. They are also working to capture physiologic responses of learners using xAPI. He hopes to discuss at the workshop in May.
Valerie questioned whether the physiologic data on learners was considered health data or educational data. David replied it is generally considered as educational data but admitted there can be a fine line. Erick asked if David is using a particular xAPI profile. David replied that Open Labyrinth uses the Virtual Patient profile we are building.
3 Ways to identify and correlate activity streams from multiple sources
David asked the group if there is a recognized way of gathering context on the scenario, user, and case from multiple streams. His developer has suggested a UUID, but its not clear who would maintain that. Ellen commented the various pieces are great individually but don’t necessarily work together. It’s difficult to get a coherent picture of the whole. Authorization, single sign on, and workflow management are all issues. David suggested storing a common scenario ID. If the UUID were long enough you would not need central registration. Ellen agreed adding connectivity profile could contain references to other activities.
Ellen commented that she was confused by the rules around looking up activity data. She asked if every provider has access to everything. Jason noted that probably isn’t realistic. He added that through OAuth scope value you can limit the visibility of an activity within the LRS, but it is not enforced and is not as robust. Domain specific rules may add more security. Ellen agreed that is not part of the roadmap currently. David suggested you could set filters and forward a set of statements.
Tom discussed the interest in launch and how you configure content. They looked at IMS LTI and CMI 5 launch as possible solutions. They found CMI 5 to be the most hopeful, but it will require some changes. They created a system that would handle launch initiation and other functions like registration ID and providing context information during launch. David commented that IMS LTI is more broadly adopted than CMI 5. Tom commented that LTI is server to server and doesn’t work for content not managed or hosted. He noted that ADL has joined IMS and spoken with them. He added that they are interested in use cases, trying to initialize content not directly linked to a server. He referred the group to ADL’s project homepage to determine what issues can be solved. He suggested the group send him or Jason direct issues for feedback. Valerie also offered to be the conduit between the group and ADL. Ellen suggested one use case might be to give an app access to an individual’s data. Tom suggested OAuth 2.0 may address that. It is not directly supported in the specification, but its use is not precluded.
4 Weighting competency scores
Ellen continued the discussion on activity profile and requirements to achieve a certain level. Valerie asked what determines if the learner meets a specific level. Ellen mentioned each activity would say record its current state: completed, passed, etc. Erick asked if completing training is a legitimate way of determining competency level. Usually a demonstration of proficiency is required. Ellen mentioned they are designing a pilot for nursing and have a real problem with onboarding tracking procedures and assessment skill level. There is no good way to insure competence. In person assessment is one component of an overall assessment.
David added we have a very broadly defined competency and he would like to see better evidence to back up judgments. Educational research has shown that detailed assessments are not as valid as global rating scales. Rating scales also introduce bias. Big data offers a more objective approach and better analysis. The key is finding a balance. Valerie suggested xAPI work might serve as a means to pull data together for people to make holistic decisions. David agreed. Valerie asked if the nurse performing a procedure is more important than how they do on a test; would those be weighted differently? David noted that artificial neural networks are better at picking out weightings and are more reliable than humans. They still require human review. Ellen mentioned that in a workshop given at last year’s conference, she and JB talked about the use of xAPI to discover intermediate competencies learners needed to work on. David agreed.
5 Profile development and next steps
Ellen spoke with Jessie about using common profile. Her work may inform ours. David asked about publishing the profile to the ADL profile registry. Tom mentioned that Jason would be happy to publish profiles. Currently there is no interface for others to make changes to the database, but the ADL staff can do that on our behalf. He noted that all IRIs should be https. They are also working to publish vocabulary terms in RDF. David asked about the timeline. Tom anticipated September for basic functionality. Valerie asked the group if they were comfortable asking ADL to publish on our behalf or feel an interface is essential to begin the process. David was comfortable if the ADL group had the time. Tom commented the publishing is the easy part. When MedBiquitous redirects to something, he needs to check with Jason.
6 DISC update
Valerie noted she had not received an update from Aaron and Megan. David, Ellen, and Valerie spoke with Aaron Silvers and Megan Bowe about the Data interoperability Standards Consortium, or DISC, which is planned to become stewards of xAPI. Valerie mentioned they are still getting off the ground and have appointed several people to their board of directors.