2 Discussion of issues in developing a profile for Mannequin Simulations - Link to working draft
Ellen noted that the American College of Chest Physicians presented at the last meeting and showed the use of very specific verbs tailored to their measurement interests, such as “blood pressured.” David commented the verbs from Chest Physicians are not generalizable. Ellen agreed we should stick with verbs that are more generic, but we may need extensions for exact measurement. She spoke with Kirsty Kitto, Senior Lecturer in Data Science, University of Technology Sydney in Australia. She recommended starting from the top down, doing rapid testing and iteration. She also recommended taking data out of JSON format and putting it in SQL. Valerie asked the reason for putting in SQL. Ellen replied that the reason given was that JSON statements are hard to use. David commented that it may be hard to di using traditional rules. Andy added that XAPI is not intended to be the end all. He anticipated a querying spec being needed, but he did not think SQL brought added benefit. A different type of tool may be needed. Valerie was surprised Kirstie would use SQL and not big data tools. Ellen clarified Kirsti’s team decided the best way to do rapid analytics was moving data out of the LRS.
Andy noted that the only querying in xAPI was around verbs and activities. There are not agreed upon practices for search and retrieval yet. There is bigger value handing it off to tools that can do bigger things. Ellen requested help building the workflow. Andy suggested finding out what the DoD is doing. He suggested starting with outcomes first. Ellen suggested considering simulations as variations of a single type. Track low-level event data and map to higher meaning across simulations to compare apples to apples. Andy agreed that made sense. David expressed concern that the top down approach made sense for contractors delivering specific tools for specific projects but not for standards development. Valerie suggested determining what the key questions are. David asked about simple tools used for elastic query of non-relational data. Andy agreed to look for data visualization and analytics around JSON. Ellen questioned what activities and extensions we should standardize to enable analysis of objectives. Valerie agreed to check with Jeff, from University of Wisconsin, about doing a pilot with some real use cases.
Andy mentioned two reasons to use standard, 1) less resources spent if a standardized approach exists already and 2) sharing results with other organizations in a meaningful way. Ellen agreed it is the same with reproducible research.
Whether it makes sense to consider several simulation types at once
The possible utility of information models
The possibility of using simultaneous profiles for each main type of agent - student, system, observer
The value of thinking backwards from possible analytics or use cases
Recording low-level event data and then higher-order mappings of that data to meaningful concepts
The need for better tooling for rapid testing purposes without having to build the real thing, etc.
Also: Russell Duhon (wrote the new profile specification) may be willing to talk to all or some of us possibly about testing profiles and learning analytics. Caveat - he is a consultant so would be interested in a paid or funded project.
- Andy will look into data visualization and analytics around JSON.
- Valerie agreed to check with Jeff, from University of Wisconsin, about doing a pilot with some real use cases.
- The group will provide input to the Teamwork Profile online document.