July 18, 2008
11 AM EDT
1. Review minutes of last meeting
Linda commented that the alliance will announce cme exchange in the almanac in September. MedBiquitous links are up; they will link to the pilot and CMEQual using suggestions.
2. Update on potential pilot projects
Tao said he was not able to convince medcases to implement. Linda commented that they have an American Heart Association project, but it will be about 8 months before they will have data. So they looked for another pilot opportunity. Allergan has supported many activities and encouraged providers to use CMEQual. They will pull together data from providers to see if standard can aggregate data from multiple providers and facilitate its distribution to supporters.
Francis asked about the timeline for the pilots. Valerie commented that given where the specification is, the pilot should be started within the next couple of months.
Pete commented that his organization, Med-IQ, is an accredited provider. They recently met with Valerie to discuss a new DVT program. They have done live events, webcasts, etc. Looking at the outcomes, they are seeing that barriers to improving prophylaxis are systemic and institution specific and therefore not appropriate for a nationwide cme initiative. They talked with Johns Hopkins about developing a program to measure dvt rates as a baseline and then work with orthopaedic surgery, general surgery, and hospitalists to come up with improvements, and then remeasure. They also thought it would be a good opportunity to pilot the metrics specification. Valerie added while the current specification addresses participation metrics and learner survey results, much of the work of this pilot would be research into the higher levels of evaluation that we need to address in version 2 of the specification. The group agreed that it was an excellent pilot opportunity.
Pete added that they wouldn't be ready to implement until early 2009. Throughout the process they will work with Valerie to keep the group informed of the objectives and progress.
Francis asked if there is a step by process describing how to implement. Valerie commented there is not, but an implementation guideline providing those steps would be helpful. The pilot experience should provide the background necessary to start working on such a document. In the interim, she asked that people contact her directly with questions about the specification. Linda added that there need to be examples others can reference, and that part of the value of cmeexchange.
Linda asked Andy if he would be willing to implement at CECity. Andy responded absolutely. Valerie recommended talking with CME ohio and Jack Kues. Andy summarized that CECity could generate member school reports using the metrics spec and add that to their reporting package. There are 7 schools involved, and some are more sophisticated than others. Andy offered to reach out to Jack. The bigger question is what will Cincinnati can do with that data.
Valerie summarized the changes to the specification. At the end of the last call, Linda had asked Valerie to incorporate simple multiple choice assessment data. Valerie added an element for reflecting that data. It uses the same basic model as the multiple choice survey questions but adds a few things. 1) a timing element to indicate if the assessment question was given before, during, or after the learning experience; 2) A classification element which uses healthcare lom element to enable evaluators to links assessment items with specific objectives, competencies, or learning outcomes; 3) an attribute to indicate if a response is correct. She also researched an existing specification for assessment scores called the Postsecondary Electronic Standards Council. Pearson and others have participated in the development of the specification, and it is used or sending standardized test scores to colleges. Valerie asked if this would be helpful to the group. The group agreed that adding scores for pre and post tests was a good idea.
4. Open discussion
Francis asked if there were any thoughts on a project to validate survey items. Valerie commented that CMExchange was certainly a step towards that, but that it may be too early. Linda agreed. They have been on the phone weekly with the alliance, which is planning to take over CMExchange in a few months. CMExchange is a clearinghouse for standards, items, assessment tools, and data. The first step is just to let people know they can submit. That is just beginning. As we get further along it may be good to get a grant in that area.
Francis asked how important it was to have pilots in different settings, for example, corporate, medical school, and specialty society. Valerie commented that having pilots in all those setting would be ideal, but getting 2 out of 3 was probably ok. Francis commented that RSNA could make a good test case and offered to talk to Mellie about implementation.
Linda asked Valerie what steps were necessary to move forward. Valerie replied that she would add a score to the knowledge assessment piece, then implementation guidelines and pilot projects were the next steps.
The next call is August 29 at 11 AM.
- Andy will contact Jack Kues regarding the pilot project
- Valerie will incorporate scores into knowledge assessment
- Those pursuing pilor ptojects will keep the group updated on their progress
- Valerie will provide support to implementers