July 15, 2009
16 BST/11 EDT/10 CDT/9 MDT/8 PDT
Attending: Tim Willet, Valerie Smothers, Rosalyn Scott, David Price, Mary Pat Aust, Maria Esquela, Matt Lewis.
1. Recap of April meeting in Baltimore
Tim reported that it was a good turnout. Tim and Rosalyn presented the working group's worked date as well as the competency environment document. The group debated a few points including assessment. There's a need to conceptualize how assessment fits into the competency environment related to the application of competencies. What we have discussed has been mostly abstract representations. The real value in using competencies is when they are applied. We'll start putting these ideas into a draft specification. Rosalyn added that assessment should be distinct from our initial work. She asked if we want to discuss further linking competence with assessment. Tim agreed we would leave assessment methods out of the definition for competency objects. But if we don't provide a way to link to record of assessment, Tim argued that we may not achieve our use cases. David and Maria agreed.
2. Overview of Medine
Tim commented that he had sent out to link to the Medine framework, which articulates a set of competencies for medical education in Europe as part of the Bologna process. The structure that Medine uses is consistent with structure that we've seen in other competency frameworks.
Tim summarized that we've reviewed 11 frameworks, mostly from medicine. Many are published by government organizations. He asked if there were additional frameworks that the group should review. David commented that the airline industry may have competency frameworks. Teamwork was critical to the landing in the Hudson, and the aviation industry's use of competencies would have implications for healthcare. Rosalyn commented that she would be working with a psychologist that has worked with the aviation industry. She offered to ask him.
3. Hospital-based competency work
Tim asked Maria to describe her work with competencies in hospital-based setting. Brea commented that when you are bringing new people on board, you need to be sure that they are competent to work with a particular patient or do a particular procedure, or if additional support is needed. This relates to the privileging process and two nurses who transfer to different departments. This nurses gain skill, they should be privileged at higher levels. Other nurses should be able to access the new nurse competency records as well. In short, the use of competency objects could help hospitals stay accredited.
Tim asked Maria how hospitals currently do this. Murray replied that they use profiles that allow programs to maintain lists they can be shared. Once a person has achieved a certain level of competence, they were able to supervise others. Tim asked if it was important to include different levels. Maria mentioned it can be represented like that; you can identify other levels or identify the procedure in a specific way. Tim questioned whether would it be a new competency or change in details of the same competency if, for example, a nurse were to complete a procedure with a different type of needle? Maria answered that they define it as a separate competency and procedure. Tim also asked Maria whether there is a separate list of competencies and Maria mentioned the hospitals' needs would dictate what competencies are. Residency review committee's may take take training competencies as well. Mary Pat agreed with the model that Maria presented, saying it was similar to the one the she had discussed previously. From a nursing perspective, there is a need to ensure the competence of the nurse and making those competencies accessible to those making assignments.
Tim asked how you determine if a person has achieved a certain competency. Maria mentioned simulations, different kinds of competency interaction, clinical performance, clinical skill experience , and assessment of performance or experience by a preceptor. Mary Pat added that new nurses may be assessed as they learn; experienced nurses may provide a demonstration or have taken a test. Tim asked if it to compare this to medicine. David mentioned he thought it would eventually be that way; some things are regulated by "local custom". There is a movement by the ABMS to align maintenance of certification with competencies. Tim thought certification of competency had more to do with attendance at CME activities or bedside procedures. David commented that all twenty-four boards are struggling with professionalism and how to measure it in a standardized fashion. 360 degree evaluations are one component, remediation may become a component in the future. David will send Valerie a link to the ABMS website with information on self audit, including practice improvement and participation in clinical registries. Addressing performance gaps will likely come next.
Maria added that there two states that allow pharmacists to write prescriptions. They have a strict system that measures competence. Tim asked Maria to distribute the list of the group.
4. Elements of a competency object
Tim continued with a discussion on the Competency Object Elements list. He is tracking consensus on the inclusion of different data elements. At the in person meeting in Baltimore, the group agreed to exclude the assessment method element. He moved onto discussion of performance criteria, which he described as specific behaviors to be exhibited as evidence of competence. Tim asked is it worth including a field expected performance criteria? David stated he would include it because it's helpful to tell folks what we want them to do. The accme is now stating learning objectives in terms of performance and outcomes. Rosalyn agreed and provided an example. A performance criteria may be appropriately placed chest tube in patient with pneumothorax. Tim asked what the competency would be. Rosalyn said that the competency would be management and pneumothorax.
Tim asked would placing a chest tube not be a sub competency. Valerie commented that was a debatable point, however, it is likely there are going to be different approaches to organizing that and she thought it was possible. Rosalyn commented we would have to be clear about doing it one way or another. Mary Pat suggested working at the lowest common denominator. Performance of the procedures the competency. But that can be rolled up into a lot of other competencies. Tim agreed that if it was a different competency object, it could have multiple parents and used in multiple ways. You may need to place a chest tube as part of many different procedures.
Tim asked if insertion of a chest tube is one performance criteria, would we be able to point to the competency directly. Valerie mentioned it is easier if listed with its own competency with a unique identifier. She mentioned there is a lot of confusion related to performance criteria. She suggested just getting rid of this and makes everything a statement.
Rosalyn mentioned a grant working with the VA and METI on woman's health increasing competency of primary care providers. The first phase is completing the didactic program on their Learning Management System and the second phase is dealing with Virtual Patient cases and demonstrating competency using Carla Pews pelvic trainer and breast models. When both parts are completed you receive a certificate. This could possibly use MedBiquitous standards including competency.
Valerie questioned whether Rosalyn would have specific assessments for each competency in the didactic and the virtual patient trainer and simulator. In such an environment using sub competencies would be ideal. She advocated for eliminating performance criteria based on the confusion that it caused. Ultimately we need to link assessment criteria and performance criteria to the competencies.
Tim continued with a discussion on competency framework for Internal Medicine residents. Would the publisher of the framework want to express that a resident performed 25 lumbar punctures with a success rate of 85% or greater? Valerie stated it's not an actual criterion for the assessment. Tim said it kind of is a criterion; there is the event of the lumbar puncture event that can be successful or unsuccessful. Mary Pat questioned how one would define what a successful lumbar puncture is. Was it the fact that you got clear fluid, used sterile technique, how is successful completion defined? David stated the recommendation can vary widely. Maria mentioned that assessment methods and performance criteria may need to be defined separately - in a database model, they would each be a separate relational table.
Valerie commented that measurable criteria are necessary to move from paper descriptions of competence to competencies serving as a technical backbone for education and assessment. She added that she recently saw the TherSim VP tool, which allows one to get detailed competency measurements based on what actions the learner takes within the virtual environment. What criteria need to be met be considered competent? Maria stated it could vary from place to place. Staffing decision could suffice, simulations would vary. Rosalyn asked if Valerie was getting too far out. Valerie replied that these criteria would not exist within the definition of the competency object but would be external. We need to begin by defining the competency object. She suggested taking out anything that resembles assessment or performance criteria from the description of the competency object. Tim agreed.
Tim, Rosalyn and Valerie will chat offline and come back to group at the next call. Rosalyn can use her grant as use case for the specification to be applied. Valerie will add a little bit to the competency document and start to develop specifications. Tim mentioned Matt and he talked yesterday and that Matt would present further information on the next call. He also mentioned if anyone had additional comments to please send them to the list serv group by email.
- Rosalyn will inquire about aviation industry use of competencies in training.
- David will send Valerie a link to the ABMS website with information on self audit, including practice improvement and participation in clinical registries.
- Maria will distribute a list of competencies related to pharmacist prescription writing privileges.
- Valerie will work with Tim to update the competency environment document based on the recent discussion and start working on a specification.