Skip to end of metadata
Go to start of metadata

Matt has prepared an analysis of using XAPI with virtual patients.

 

Some potential use cases that would link in with using XAPI for reporting. How these VP's might be actually shared in practice is an authorization and Content Brokering Issue. 

XAPI01 - Sharing a case from Clinical Practice with an Educational Institution  

XAPI02 - Sharing a case written by an Education Institution with Clinical Practice 

XAPI03 - Reporting on Virtual Patient Type Activities from systems that do not support the standard

XAPI04 - Analyzing the effect of a Virtual Patient Case in context

XAPI05 - Recording a group of learners playing through a VP collaboratively and linking that video to learning record

XAPI06 - Videoing an individual's progress through a VP for qualitative analysis 

XAPI07 - Playing a series of linked VP cases over a period of time

XAPI08 - capturing and analyzing the learning context around a VP case

XAPI09 - tracking a change to recommended practice and invalidating competency record

 

Possible basis for VP specific community profile   

Verbs 

Activities

  • No labels

3 Comments

  1. Thanks for putting this together Matt! It's very informative regarding the nuts and bolts of XAPI. In trying to go off mute and comment at today's call, I accidentally disconnected (smile)  I'm quite interested in TinCan/XAPI as a method for logging VP player activity.  Have you thought about what verbs would be valuable for our community to track?

    While I like the XAPI concept, verb definition is key. I doubt the current 15 will meet our needs. If we can agree on verbs and extensions, then I think it makes sense for the team to devise a VP XAPI best practice/standard.  In this way, assessment can be driven from the many LMSes that are planning support for access to the XAPI LRS. In it's simplest form, we could just push the series of player decisions as they walk a linear or branching narrative. Of course, ideally we'd want to match this with correct/incorrect responses. Do you think this could fit into the XAPI model?  Also for a branching narrative where the player has the opportunity to "self correct" (or recover from earlier mistakes), how do we record this outcome? In essence there would also need be a meta assessment of whether they succeeded or failed the whole scenario.  In many of our simulations the player can "succeed" in saving the VP, but fail to follow the golden path in getting to that "success" state.

    For our virtual environments, we're dumping a LOT of log data. The intent, as you suggested, is to build a log filter to extract the interesting bits for delivery to a dashboard (both for the educator and learner). I can envision also sending a filtered feed of XAPI tuples to an LRS. Our Big Data and informatics folks are eager to get their hands on the log file stream to tie user behavior in game to outcomes. We still don't fully understand HOW we will use all of this data for assessment or feedback loops for the learner, thus I'd vote for capturing everything as an option for a VP XAPI "best practice". Then as we become more experienced about what is important, filters can be created to present these insights to the learner and educator.

    I think/hope XAPI has enough momentum that several companies will build "viewers" for LRS data. Given that, we'd want to support standard verbs at the "wrapper" level, and then hopefully XAPI's architecture will allow the "viewers" to import the VP XAPI standard to provide the drill down data. Just like Khan Academy's analytics tools, I anticipate educators will primarily want to drill down on learners having difficulty. Moreover, we might point to WHERE in the experience the learner struggled and what their decision points/activities were at that stage.  This may be a stretch for the XAPI, but is a vision worth striving for.

  2. Hi Micheal, 

    I was late to the call as i couldn't get DTMF dialing on my phone to work, so i am relieved its not just me who hits the wrong button at the vital moment (wink) glad you found the XAPI stuff useful. 

    My starting point with verbs was to see if the standard set from ADL were 'good enough', but  the XAPI spec is quite clear that verbs should have the same meaning, so taking a generic one and having a VP related meaning is contrary to that. Which suggests quite strongly the need our own, assuming that as community we can agree on a set and their consistent meanings! As to the actual verbs, the answer is not yet, but after yesterdays call its definitely on my TODO list. We have implemented feedback rules in quite a big way so I will start with those and work backwards. Would it be useful if i posted them here as item for discussion? 

    Assuming we go down the road of a VP Specific Profile then I think we could also have a VP particular set on interactions rather than just 'Simulation'. So if we had 10 interactions and and 15 verbs that might give us up to 150 combinations instead of having to define 150 verbs. I currently think using XAPI for a straight forward play through, linear or branching is very feasible (and capturing true/false, scored responses). Self-correcting and succeeding without doing it 'properly', are a more interesting challenge and one that has pointed me back to the feedback rule mechanism that we use.  

    As I said on the call, one of the reasons for my interest in XAPI is we still struggle with what it 'means' to play a Virtual Patient case. They can be a lot more sophisticated/subtle than a set of MCQ's, which is why I got interested in them in the first place.  For example, they are great for provoking an ethics discussion but mainly inappropriate to assessing the consequent discussion. As part of our ongoing attempts to close the loop we've just purchased MORAE http://www.techsmith.com/morae.html to do some qualitative work with how our learners use cases and relate it back to their 'journey' through a case. I am also keen for someone with an analytics background to do quantitative work as well and 'fishing' for someone to run/fund a longitudinal study to see if there is any actual long term change in practice due to VP use. I agree that having an LRS populated to the gunnels with all possible information about learners journeys' is a precursor to the HOW. From what i understand, the big data crowd  let their clever algorithms spot the pattern and then work out what question the pattern is answering.

     The more familiar I get with XAPI the more I like it. The idea of decoupling the tracking info from the content is inspired and as you say should lead to slew of viewers with interesting features and capabilities. Which brings us back to the range of use cases that could be supported by such systems. If you interested I am happy to share these as we identify them on this wiki ?


     



  3. Please do share them, Matt! And everyone else, too. (smile)