Child pages
  • Performance Framework - Use Cases
Skip to end of metadata
Go to start of metadata

Use Case Titles

  1. Map performance frameworks to competencies, including Entrustable Professional Activities
  2. Publish a performance framework for use in curriculum planning and assessment
  3. Import descriptions of performance levels for use in assessment
  4. Reference a Performance Framework for use in assessment
  5. Describe an individual’s current level of performance for purposes of formative or summative assessment
  6. Describe an individual’s level of performance over a longitudinal period for purposes of formative or summative assessment

  7. Associate assessment evidence with a particular level of performance in a portfolio
  8. Define where in a curriculum students are expected to achieve certain levels of performance
  9. Define what level of performance is required to progress to the next phase or block within a curriculum

  10. Use in a system that is capable of showing a learner changes in performance over time
  11. View data regarding the performance levels of learners in a program for purposes of program evaluation (external)
  12. View data regarding the performance levels of learners in a program for purposes of program evaluation (internal)
  13. Publish program data for public viewing

Actors

AdministratorA person that administers an assessment system, curriculum management system, portfolio system, or performance management system that uses one or more performance frameworks. 

Assessment system

A system providing tools and a database for managing learner assessment data. 
Curriculum developer An individual or organization developing the structure and content of an educational program.

Curriculum management system

A system providing tools and a database to manage the structure and planning for a curriculum for an educational program. 
Decision maker An individual or organization that makes a decision with regard to academic or career advancement of the learner. 

Evaluator

An individual or organization that reviews individual learner data as part of an education or assessment process.
External Program Evaluator An individual or organization reviewing a learning program for evaluation purposes. An accrediting organization may be an external program evaluator.
Internal Program Evaluator An individual or organization reviewing a learning program for purposes of internal program evaluation and improvement.  

Learner

An individual engaged in study, training, or self assessment.  

Mentor An individual who reviews learner data as part of an ongoing formative feedback or to mentor the learner in their professional formation. 
Performance framework developer An individual or organization that develops a set of performance levels associated with a competency framework.  

Performance management system

A system providing tools and a database to manage learner performance data. 

Portfolio system

A system providing tools and a database where the learner can manage their own achievement data and resources.
Program administratorAn individual or group of individuals that administer a learning program.  
Program evaluation systemA system that allows external program evaluators to review program evaluation data from multiple educational programs. 

 

 

Map performance frameworks to competencies, including Entrustable Professional Activities

ID

PF10

Title

Map performance frameworks to competencies, including Entrustable Professional Activities

Actors

Framework developer

Trigger event

An organization wants to map performance frameworks to competencies, including Entrustable Professional Activities.

Success End Condition

The performance framework captures how the framework maps to competencies, including Entrustable Professional Activities.

Assumptions

The framework developer has developed a framework of performance levels related to a specific competency or competency framework.

Description

The framework developer maps how the performance framework relates to competencies and competency frameworks (including Entrustable Professional Activities).

The framework developer translates that mapping, along with the performance levels, into a machine readable, specification conformant performance framework document.

Transactions

Converting the performance framework, including mappings to comeptencies, into a conformant XML document.

Exceptions

The performance framework is not accessible, not readable, not valid, or the mappings are misrepresented.

Potential Standards

Performance Framework

Publish a performance framework for use in curriculum planning and assessment

ID

PF20

Title

Publish a performance framework for use in curriculum planning and assessment

Actors

Framework developer

Trigger event

An organization wants to publish a machine-readable, specification conformant performance framework.

Success End Condition

The performance framework is available for download or referencing.

Assumptions

The framework developer has developed a framework of performance levels related to a specific competency or competency framework.

Description

The framework developer translates the human readable performance framework into a machine readable, specification conformant performance framework document.

The framework developer posts the machine readable performance framework document on the Internet.

Transactions

Converting the performance framework into a conformant XML document.

Exceptions

The performance framework is not accessible, not readable, or not valid.

Potential Standards

Performance Framework


Import descriptions of performance levels for use in assessment

ID

PF30

Title

Import descriptions of performance levels for use in assessment

Actors

Framework developer, Administrator , Assessment system, Performance Management System, Curriculum Management System, or Portfolio System

Trigger event

An organization wants to download and use a performance framework developed by another organization.

Success End Condition

The performance framework is available for use in learner assessment systems.

Assumptions

The framework developer has developed a framework of performance levels related to a specific competency or competency framework. The performance framework is available as a machine readable, specification conformant document.

Description

The administrator downloads the performance framework or otherwise obtains a machine readable, specification-conformant version.
The administrator imports the machine-readable performance framework into the Assessment system, Performance Management System, Curriculum Management System, or Portfolio System.
The system stores the performance framework and enables administrators, evaluators, mentors, decision makers, and learners to begin working with it (i.e. defining learner’s current performance levels and providing a roadmap for future learning).

Transactions

Importing the performance framework into the assessment system.

Exceptions

The performance framework is not successfully imported.

Potential Standards

Performance Framework

 Reference a performance framework for use in assessment

ID

PF40

Title

Reference a performance framework for use in assessment

Actors

Framework developer, Administrator , Assessment system, Performance Management System, Curriculum Management System, or Portfolio System

Trigger event

An organization wants to reference a published performance framework.

Success End Condition

The performance framework is available for use in an Assessment System, Performance Management System, Curriculum Management System, or Portfolio System.

Assumptions

The framework developer has developed a framework of performance levels related to a specific competency or competency framework. The performance framework is published on the Internet as a machine readable, specification conformant document.

Description

The administrator configures the Assessment System, Performance Management System, Curriculum Management System, or Portfolio System to use the performance framework by entering the URL of the machine readable, specification-conformant performance framework document. 
The system accesses the performance framework and any linked documents, enabling administrators, evaluators, mentors, decision makers, and learners to begin working with it (i.e. defining learner’s current performance levels and providing a roadmap for future learning).

Transactions

Accessing the performance framework.

Exceptions

The performance framework is not accessible or not readable.

Potential Standards

Performance Framework

 


Describe an individual’s current level of performance for purposes of formative or summative assessment

ID

PF50

Title

Describe an individual’s current level of performance for purposes of formative or summative assessment

Actors

Evaluator, learner, assessment system

Trigger event

The learner is being assessed by an evaluator

Success End Condition

The learner’s level of performance is captured electronically in an assessment system or performance management system in a manner that conforms with the performance framework being used by the organization.

Assumptions

The organization has an assessment system or performance management system with access to the performance framework in a machine readable format. Assessment instruments have been developed based on the performance framework.

Description

Evaluator accesses an assessment instrument in an assessment or performance management system.
Evaluator uses the assessment instrument to indicate a level of performance (or multiple performance levels related to multiple competencies) for a learner.
The system stores the performance level data for the learner.

Transactions

 

Exceptions

Evaluator is unable to access the instrument or record the results.

Potential Standards

Performance framework
Competency framework
Educational Achievement

Describe an individual’s level of performance over a longitudinal period for purposes of formative or summative assessment

ID

PF60

Title

Describe an individual’s level of performance over a longitudinal period for purposes of formative or summative assessment

Actors

Evaluator, decision maker, learner, assessment system

Trigger event

The learner reaches a transition point that requires a longitudinal assessment.

Success End Condition

The learner’s level of performance is captured electronically in an assessment system or performance management system in a manner that conforms with the performance framework being used by the organization.

Assumptions

The organization has an assessment system or performance management system with access to both the performance framework and the learner's performance assessment data in a machine readable format.

Description

An evaluator or decision maker accesses the learner's performance assessment data in an assessment or performance management system.
Evaluator or decision maker reviews the learner's performance assessment data and uses that data to inform a longitudinal assessment of the learner.
Evaluator or decision maker uses the assessment or performance management system to record the performance level for the learner for the longitudinal period. 
The system stores the performance level data for the learner.

Transactions

 

Exceptions

Faculty is unable to access the learner's performance data or record the learner's performance level.

Potential Standards

Performance framework
Competency framework
Educational Achievement


Associate assessment evidence with a particular level of performance in a portfolio

ID

PF70

Title

Associate assessment evidence with a particular level of performance in a portfolio

Actors

Evaluator (or mentor?), learner, Portfolio system

Trigger event

Evaluator is reviewing evidence in a learner’s portfolio

Success End Condition

The learner’s level of performance with regards to a specific piece of evidence is captured electronically in a portfolio system in a manner that conforms with the performance framework being used by the organization.

Assumptions

The organization has a competency-based portfolio system leveraging the performance framework.

Description

Evaluator accesses a competency-based learner portfolio.
Evaluator reviews evidence stored in the portfolio and associates a particular level of performance with that piece of evidence.
The  portfolio system stores the data regarding the association of the evidence to a particular level of performance.

Transactions

 

Exceptions

Evaluator is unable to access the portfolio or associate the evidence with a performance level.

Potential Standards

Performance framework

Educational achievement

 


Define where in a curriculum students are expected to achieve certain levels of performance

ID

PF80

Title

Define where in a curriculum students are expected to achieve certain levels of performance

Actors

Curriculum management system, curriculum developer, administrator

Trigger event

The curriculum developer integrates performance level expectations into the curriculum.

Success End Condition

The curriculum management system contains data on expectations for learner performance.

Assumptions

The organization has a curriculum management system capable of integrating  performance frameworks.

Description

Administrator accesses the curriculum management system.
The Administrator associates specific performance levels with parts of the curriculum, indicating where students are expected to achieve certain levels of performance.

Transactions

 

Exceptions

Administrator is unable to access the curriculum management system or associate part of the curriculum with a performance level.

Potential Standards

Performance framework

 

Define what level of performance is required to progress to the next phase within a curriculum

ID

PF90

Title

Define what level of performance is required to progress to the next phase or block within a curriculum

Actors

Curriculum management system, curriculum developer, administrator

Trigger event

The curriculum developer integrates performance level progression requirements into the curriculum.

Success End Condition

The curriculum management system contains data on performance level requirements for progressing through the curriculum.

Assumptions

The organization has a curriculum management system capable of integrating performance frameworks for performance-based progression.

Description

Administrator accesses the curriculum management system.
The Administrator associates specific performance levels with parts of the curriculum, indicating that they are required for progression to the next phase or block.

Transactions

 

Exceptions

Administrator is unable to access the curriculum management system or associate part of the curriculum with a performance level.

Potential Standards

Performance framework


Use in a system that is capable of showing a learner changes in performance over time

ID

PF100

Title

Use in a system that is capable of showing a learner changes in performance over time

Actors

performance management system, administrator, mentor, decision maker

Trigger event

A learner mentor, or decision maker wants to review a learner's progress over time.

Success End Condition

The performance management system is able to display longitudinal data on learner performance.

Assumptions

The organization has a performance management system capable of accessing or storing performance data and displaying it to learners mentors, and decision makers.

Description

The learner, mentor or decision maker accesses the performance management system.
IF a mentor or decision maker is accessing the system, the mentor or decision maker selects the learner to review.
The learner, mentor, or decision maker selects a competency for which he or she would like to view performance data.
The performance management system compiles and displays performance data related to the selected competency.

Transactions

Transfer of performance data to a performance mangement system.

Exceptions

The performance management system is unable to compile the full record of performance data,

Potential Standards

Performance framework
Educational achievement

 

View data regarding the performance levels of learners in a program for purposes of program evaluation (external)

ID

PF110

Title

View data regarding the performance levels of learners in a program for purposes of program evaluation

Actors

External program evaluator, performance management or assessment system, administrator

Trigger event

The educational program undergoes an external program review, potentially as part of a program accreditation process.

Success End Condition

The external program evaluator has access to program data on learner performance.

Assumptions

The educational program has a performance management or assessment system capable of sending data to the External program evaluator in a machine readable, specification conformant format.

The External program evaluator has a Program evaluation system capable of importing aggregate learner performance data from the educational program's performance management or assessment system.

Description

The administrator accesses the performance management system.
The administrator uses the system to initiate the transfer of aggregate learner performance data to the program evaluation system.
The external program evaluator accesses the program evaluation system to view data about the aggregate performance of learners in the educational program.
The external program evaluator uses the aggregate performance data to make decisions and recommendations with regards to the educational program.

Transactions

Transfer of aggregate performance data to a program evaluation system.

Exceptions

The program evaluation system is unable to read the aggregate learner performance data.

The performance management or assessment system is unable to send data to the program evaluation system.

Potential Standards

Performance framework
Aggregate educational achievement data

 

View data regarding the performance levels of learners in a program for purposes of program evaluation (internal)

ID

PF120

Title

View data regarding the performance levels of learners in a program for purposes of program evaluation (internal)

Actors

Internal program evaluator, performance management or assessment system

Trigger event

The educational program undergoes an internal program review

Success End Condition

The internal program evaluator has access to aggregate program data on learner performance.

Assumptions

The educational program has a performance management or assessment system capable of presenting aggregate learner performance data.

Description

The internal program evaluator accesses the performance management system.
The internal program evaluator uses the system to develop reports on aggregate learner performance data.
The internal program evaluator uses the reports to make decisions with regards to the educational program.

Transactions

-

Exceptions

The program evaluation system is unable to compile aggregate learner performance data.

Potential Standards

Performance framework
Aggregate educational achievement data

 

Publish program data for public viewing

ID

PF130

Title

Publish program data for public viewing

Actors

Program administrator, performance management or assessment system

Trigger event

The educational program wishes to/is required to publish its performance data.

Success End Condition

The public is able to view aggregate program data on learner performance.

Assumptions

The educational program has a performance management or assessment system capable of presenting aggregate learner performance data.

Description

The Program administrator accesses the performance management system.
The Program administrator uses the system to develop reports on aggregate learner performance data.
The Program administrator publishes the reports in a publicly accessible venue.

Transactions

-

Exceptions

The program evaluation system is unable to compile aggregate learner performance data.

Potential Standards

Performance framework
Aggregate educational achievement data

  • No labels

2 Comments

  1. Based on the discussion at the MedBiq conference:

    • determine when someone can operate without supervision; or act as a supervisor
    • practicing MDs demonstrate maintenance of competence for license renewal
    • merge data from multiple sites and rotations into an overall competency progress report
    • tie CPD performance assessments to competencies
    • surgery: general EPAs for all grads plus additional EPAs for specific procedures
    • manage tasks at the 'top of one's license' while delegating more basic or advanced tasks to other professionals
    • EPAs/credentials/competencies/performance may apply to teams, not just to individuals
    • Patient profile (i.e. diagnoses, condition) generates a task list; tasks are then assigned to appropriate persons
    • integration with quality measurements
    • reward individuals when they meet certain performance criteria
    • eventually a person will have to view data and make a judgment
    • longitudinal tracking of performance - level changes over time
  2. Steve Clyman astutely points out that the curriculum may be EPA-based, in which case the phase or block in the curriculum would have a set of EPAs associated with it. Entrustment of those EPAs would be required in order to progress to the next phase/block of the curriculum.

    Entrustment and performance level decisions may be based on multiple learner data points.

    This is an excellent use case for our Curriculum Inventory work.