Skip to end of metadata
Go to start of metadata

CI Workgroup Recording (October 31, 2019)




12-1pm EST


WebEx (Please use video if available)


PowerPoint, Handout


Johmarx Patton, Jeff Kaminski, Hugh Stoddard, Sascha Cohen, Susan Albright, CI Workgroup members



* Expected absence ** Attendance tentative *** Late arrival anticipated ****Guest/Delegate


Review requested revisions to CI standard


 Topic and Notes from Meeting

Action Inform/Discuss/Decide)




Brief introductions of MedBiquitous staff and Workgroup leads



5 min

  • Johmarx Patton: AAMC, Director of Educational Technology and Standards
    • Involved with MedBiquitous for the past six years.
  • Jeff Kaminski: AAMC, Medical Education Operations Specialist
  • Hugh Stoddard: Emory University, School of Medicine, Assistant Dean for Medical Education and Research
    • Involved with both MedBiquitous and AAMC Curriculum Inventory
  • Sascha Cohen: UCSF, Director of Technology Strategy and Development for Medical Education
    • Involved with MedBiquitous and Curriculum Inventory for many years – especially since traction began in 2015
    • Also, Director of the Ilios project’s Curriculum Management platform, which is a vendor tool allowing schools to upload to the curriculum inventory.
  • Susan Albright: M:ed:Integrate, Owner


Review of procedures for revisions



5 min


  • To develop XML standards for the exchange of curriculum data for benchmarking and educational research.
  • Impetus was a request in August 2019 from Angela Blood, AAMC Director of Curricular Resources.

Review of Procedures and the Process for Revisions:

  • During the interim before ANSI accreditation is renewed, MedBiquitous is still following the ANSI essential requirements.
  • Request for revisions received from Angela Blood.
  • Requests reviewed by Johmarx Patton, MedBiquitous’ Technical Steering Committee, and CI Workgroup (CIWG) leads, then shared with full CIWG for feedback. The period for CIWG members to share input lasts until November 15, 2019.
  • After November 15, requests will be posted for public input for approximately six weeks.


Procedure for decisions that remain after virtual meeting and public open call for feedback



5 min

Once public input received and incorporated, the final package will be presented for vote.


Highlight of key decisions to be made, with input from participants



40 min

Decision #10: Primary instruction method vs. time per method; Page number 27

  • Description: The data items “instructional method” and/or “Assessment method” are required for events. Schools can indicate more than one method (or repetition of the same instructional method) but must indicate which method is “primary.”
  • Current State: Vendors who use a scheduling/calendaring function to populate their CI may find this change relatively easy; vendors who do not may find this change challenging. A business rule implication may be whether or not to require schools account for 100% of time in events with various instructional methods.
  • Proposal for Change: Remove the concept of “primary” and replace instead with “amount of time” (hours, minutes) per method tagged for instruction, assessment, and resources. Keep the ability to tag multiple instructional and assessment methods to a single event, including repetition of the same method.
  • Options:
    • No change
    • Remove attribute “primary” and replace with attribute “amount of time”
    • Keep attribute “primary” and add attribute “amount of time”
  • Discussion:
    • The current model is such that any event may take any number of instructional or assessment methods (e.g., An event could be flagged as lecture, assessment, and/or small group discussion; however, one must be flagged as the primary and the entire time allotted for that event must be logged for the primary activity. Logging as instructional or assessment is determined by the primary flag.) “Primary” flag was originally created and applied to the standard to circumvent micromanaging the allotment of time for multiple instructional methods within a given event. This approach can create a variable of weighting to individual activities, and thus variable and inadequate quality to the data received in the system.
      • Current language does not specify as an “optional” attribute; however, if stated as a “required” attribute other issues would arise.
      • If a change was made, could it be a hybrid approach: keep as-is, but allow participants to opt for the new route if desired?
      • From the developer perspective, this is a lot of work to undertake. (i.e., re-do database and interface, transition all existing data to new model, and then ask clients to re-do all their work, too.)
      • Support expressed for adding attribute of time as optional, with awareness of work/time that would be involved. Idea: Include in the event ID a separator to track the multiple parts of that event. This would not change the data structure while allowing to gather additional information.
      • Questions: Does a school/institution look at an “event” or an “activity” as a unitary item OR does it look at an action of a particular type within a curriculum as a unitary item? How do we want to address the passage of data?
      • MedBiquitous standards should be kept for universal purpose and use by multiple organizations.
      • Clarification made that “Amount of Time” would be optional.
      • Clarification made that, from the AAMC CI perspective, no information is dropped regardless if denoted as primary or not. However, in the PDF verification report only time allotted to primary is captured, and that is what curriculum participants review.
      • Additional information and discussion needed on this topic before making a decision.
      • Question for the workgroup to consider: How would one indicate that students are participating in an activity (e.g. simulation), when the assessment is of their participation over the entire time?
  • Decision: Circle back with the AAMC CI team for further review of this particular request.

Decision #13: Expectations/Competency Objects; Page number 30

  • Current: “Expectations” encompass many concepts: objectives, competencies, learning outcomes, milestones, performance levels, EPAs. Expectations and Competency Objects essentially mean the same thing: learning objectives, although both terms are used in the CI specification documents. It is confusing for users of the specifications to have multiple terms within a document with the same meaning.
  • Options:
    • No change
    • Change Expectations to Learning Objective
    • Preferred option: No change
  • Discussion:
    • Changing “Expectations” to “Learning Objective” narrows, rather than expands, the capacity for the standard to address a variety of institutions and situations.
    • From an educator’s perspective, they are not theoretically the same thing and the impact of how they are used is not the same thing. While in practice they may be approached as synonyms, this is not appropriate. By narrowing, this would promote incorrect use of the terms. Originally, the term “Expectations” was chosen because it was generic, so institutions could use as desired.
    • Questions: How do we differentiate in situations where “Expectations” is being used interchangeably with “Learning Objectives” or for things that are not competencies or milestones? Does our role also involve training users how to appropriately apply educational theory?
    • The term “Expectations” contains the elements of objectives, competencies, learning outcomes, milestones, performance levels, and EPAs. And, it depends how users are applying those terms at their respective institutions. As a higher-level term, it does uphold and provide a framework for the standard.
  • Decision: Preferred option is no change.

Decision #6: Profession/Specialty

  • Current: The data items “Profession” (the appropriate response is “physician” for all AAMC CI submissions) and “Specialty” (e.g., anesthesiology, cardiology, etc.) currently are tagged at the program level from the MedBiquitous Healthcare LOM specifications.
  • Proposed: Remove the data items “Profession” and “Specialty” from the AAMC CI specification requirements.
  • Rationale: Tagging at the specialty level isn’t helpful to AAMC or our members. It would reduce the number of documents CI administrators and users need to reference and reduce AAMC staff burden to ensure documents and website links are up to date. In addition, this item for “physician” is redundant data for AAMC with the data item “MD.”
  • Options:
    • No change
    • Remove from standard
    • No change; provide guidance on excluding data from upload
    • Preferred option: No change; provide guidance on excluding data from upload
  • Discussion:
    • Question: Is the issue with the LOM standard?
      • Clarification: It is the CI standard, but it extends the LOM standard by taking elements from it for MedBiquitous’ purpose.
    • With regard to the rationale, concerns raised:
      • The terms “physician” and “MD” are not necessarily the same. “Physician” includes DO, MBBS, etc. – not just MD.
      • This request is too institution-specific. The standard should be equally representative of all health professions.
      • Dropping the LOM from this standard doesn’t promote any value, rather it removes the opportunity to provide additional information.
    • The standard needs to represent all health professions.
    • As per the Preferred Option, an offer was extended to provide guidance on how to exclude unneeded date from the upload.
  • Decision: Preferred option is no change; provide guidance on excluding data from upload

Decision #4: Remove Data item “Supporting Link”; Page 16

  • Description: The supporting link field allows schools to provide a web address that could contain documentation on their curricular structure. This data item is optional.
  • Current State: The AAMC does not use the supporting link field and cursory testing suggests many of these links are non-functional. E.g., 404 Page Not Found.
  • Committees to Know - Examples: “Page Not Found,” “Live Page,” “Home Page.”
  • Options:
    • No change
    • Remove from standard
    • No change; provide guidance on excluding data from upload
    • Preferred Option: No change; provide guidance on excluding data from upload
  • Discussion:
    • The consensus from workgroup leads is that any time a request is made to remove a data attribute from a standard there will be hesitation.
    • A suggestion was made to create best practices for the breadth of current use for all the standards. This standard could be the first endeavor in that direction.
  • Decision: Preferred Option: No change; provide guidance on excluding data from upload.

NOTE: Due to time constraints, workgroup leads decided to expedite the process by reading each proposed change and then allowing a moment for voices of dissent. If no dissent expressed, then the decision was ratified as No Change.

Decision #21: Preconditions & Postconditions; Page 40

  • Description: Describes preconditions (e.g., prerequisite) that must be met for a learner to take this course. Describes how the curriculum changes depending on the learner’s performance in the sequence block. These are optional attributes of all sequence blocks. This is an open-text field.
  • Current State: In the 2017-2018 data, three schools used the precondition field, none used the postcondition field. AAMC has not used data from this standard for reporting.
  • Options:
    • No change
    • Remove from standard
    • No change; provide guidance on excluding data from upload
    • Preferred Option: No change; provide guidance on excluding data from upload.
  • Discussion:
    • Question: Is there enough data currently provided to understand what should be included in the dropdown field? No. That would be a challenge technically. To allow that controlled vocabulary to be addressed as a variable by individual schools could become ineffective.
  • Decision: Preferred Option: No change; provide guidance on excluding data from upload.

Decision #5: Remove data item “Educational Context”; Page number 21

  • Description: The data item “Educational Context” specifies what type of educational program it is (e.g., patient education, primary education, etc.) It is an optional field.
  • Current State: AAMC does not use the data from this item.
  • Proposal for Change: All AAMC’s submissions are coming from the same educational context. For MedBiquitous, it seems likely to create some discrepancy in quality in any case, as many of our schools might call themselves “undergraduate education” programs, but this might overlap with nursing schools with undergraduate bachelor programs. It may be that all medical schools are meant to be considered “undergraduate professional education” but it is unclear.
  • Options:
    • No change
    • Remove from standard
    • No change; provide guidance on excluding data from upload
    • Preferred Option: No change; provide guidance on excluding data from upload.
  • Decision: Preferred Option: No change; provide guidance on excluding data from  upload.
  • Decision: No change; provide guidance on excluding data from upload

Decision #8: Interprofessional; Page number 25

  • Description: The data item “Interprofessional” is an optional.
  • Current State: The placement of this data item as an attribute of an event is odd. There are other data items like “clinical environment” or “service-oriented” that also could be considered attributes of an event but are instead included in the AAMC/MedBiquitous standardized instructional methods. Currently the AAMC does not use this item for reporting results; more school training would be needed to have people use this item consistently in their data sets.
  • Proposal for Change: To be determined. This data item should be reviewed with representative members and vendors. It should not be continued “as is.”
  • Options:
    • No change
    • Remove element
    • No change, item optional; provide guidance on excluding data from upload
    • Preferred option: No change, item optional; provide guidance on excluding data from upload
  • Discussion:
    • This particular decision is different than the preceding ones because:
    • “Interprofessional” is not the same as “clinical environment” or “service-oriented”
    • “Service-oriented” is not in the vocabulary
    • All asks are not identical; some are more involved than what may appear on the surface.
    • The interprofessional community continues to struggle with the nature and boundaries of what their own definition will be. However, this is not a reason to remove it.
    • Question: How are we providing training and support for constituents using the standards? The breadth and depth of resources shared with the community should be addressed, such as training, support, and best practices, so there is better understanding to what an item is and how to apply it.
    • Better education is needed to teach people/institutions how to approach the data.
    • Question: Are there other attributes that require additional measurement and evaluation? This is an opportunity for a larger conversation to develop.
  • Decision: No change, item optional; provide guidance on excluding data from upload

Decision #14: Expectation IDs; Page number 30 (Competency Object Framework – pages 17-18)

  • Description: Expectations at each level (e.g., program, sequence block, etc.) are assigned an ID by AAMC.
  • Current State: There is not a field that explicitly asks for expectation IDs which schools have assigned to their expectations. This is a numeric value that has no meaning to schools so isn’t valuable to share back. Schools may develop schema for creating their own expectation IDs that have meaning, and would be valuable for AAMC to share back with them in reports. Some schools are currently using the “description” field for this purpose, but others have evolved using the description field for other things, and as these uses may have meaning to schools, we’re likely to get negative feedback if we try to relabel “description” as “ID,” rather than creating a new optional field.
  • Proposal for Change: Create a new optional field for expectation ID per expectation at each level (e.g., program, sequence block, event).
  • Discussion: There was not enough time to address discussion points.

NOTE: In the effort to respect time, moving forward CI workgroup leads will meet in advance to review Decisions. Any items requiring additional deliberation will be brought before the entire workgroup for discussion. PowerPoint slides and pertinent materials will be shared in advance for review.

Action Items from This Meeting:



Create best practices for the breadth of current use for all the standards. Standard #4 could be the first endeavor in that direction.


In progress

Share next steps and a revised timeline


In progress

  • No labels