Use Cases for MEMS and Proposed Format for User Evaluation



Activity metrics will need to be communicated to activity supporters for several different kinds of events. Initially, we propose supporting metrics reports for in-person, single event CE activities, online CE courses, print CE activities, and aggregations that combine these individual activities. Self reported CE activities are considered out of scope for the first iteration of MEMS. All metrics will be aggregate data, not individual data. The standard will allow extensions so that evaluation data beyond the core set identified in this document may be communicated to other organizations. Most elements will likely be optional; some extremely common ones may be required if the committee finds it appropriate. This specification is not intended to encapsulate all metrics data that will be maintained in an organization’s internal systems; rather, it is intended to encapsulate the key quality metrics that may be transmitted to an outside organization interested in collecting data on the quality of educational activities.


Use Case 1: Reporting Metrics for an Online CE Activity for a Specific Report Interval

In this use case, the CE provider needs to provide evaluation and metrics information regarding an online activity. These reports will cover a stated period of time, defined in the report. Website usage data (registered user, unique visitors, course completion rate) are relevant.


Use Case 2: Reporting Metrics for an In-Person CE Event

In this use case, the CE provider needs to provide evaluation and metrics information for an in-person event or meeting. The report will cover one event only.


Use Case 3: Reporting Metrics for a Print-based CE Activity

In this use case, the CE provider needs to provide evaluation and metrics information for monographs, journal-based CE, and other print-based educational activities. The report will cover one print activity only.


Use Case 4: Reporting Metrics for Several Related Activities

Many educators will offer different modalities to teach the same learning objective or will offer a multifaceted approach that combines print, live, and online activities around a single topic. In this use case, the provider needs to report metrics for each individual activity as well as metrics relevant to the group of activities, such as how many learners participated in multiple activities. The initial version of MEMS will provide a way to link related activities at a minimum.


In all use cases, reporting on numbers of participants, completion rates, feedback, credit units awarded, user evaluations, professions and specialties of participants, among other data are relevant.


These reporting metrics may also be relevant to CE providers, accreditors, and activity certifiers. However, these constituencies have not yet fully defined their requirements.


Participant Activity Evaluation

The participant activity evaluation section addresses the participants’ evaluation of the educational activity and may include the following types of information.



For each of these elements using the Likert scale, we propose using a 5-point likert scale. The number of participants responding to each possible answer would be included.



To track perception of commercial bias perceived and other yes/no questions, the number of participants responding yes as well as the number or respondents stating no would be reported..



Participation Metrics


A section on participation metrics would include data on the number of learners participating in the activity and the extent of their participation [VCS1] .



To indicate the number of credits awarded or available, we would use the same credit format as used in MEDBIQ-SCORM, which allows one to indicate the type of credits, the activity certifier, provider accreditation, and the unit of measure.



The activity length element communicates the duration of the activity. If this is an online or print activity, this represents an estimate (or should this be an average?). This may include years, months, days, hours, minutes, and seconds. A typical activity would likely use only hours and minutes.

[VCS1] 1 Added NumberOfRegisteredParticipants, NumberOfParticipantsToReceiveMaterials, NumberOfParticipantsAttending, NumberOfDistinctHostsOrVisitors. Removed NumberOfParticipantsToComplete and NumberOfParticipantsFailingAssessment.