September 24, 2013
10 AM PDT/11 AM MDT/12 PM CDT/1 PM EDT
Attendees: Francis Kwakwa, Chair; Doris Auth, Tony D’Ambrosio, Jennifer Baumgartner, Mark Baczkowski, John Fitzgerald, Ilana Hardesty, Brad Hill, Cynthia Kear, Linda Kitlinski, Joanna Krause, Lauren Long, Adam Martin, Tom McKeithen, Rajesh Patel, Valerie Smothers, John Sweeney, Amy Tan, Dimitra Travlos, John West, and Julie White.
1 Review minutes of last call
The minutes were approved as submitted.
2 Results of RPC user Acceptance testing
Raj described the results of the RPC user acceptance testing. He explained the system was developed to support the collection of data for the first reporting period. Internal RPC team members acted as accreditors and tested many scenrios, including missing required fields, duplicate data from an accreditor, and duplicate data from multiple accreditors. The system handled the data submissions accurately. The system flagged each of the records from the same accreditors, or multiple accreditors, and when the record was flagged they reached out to the submitter and corrected the error. They developed functionality so the team could perform analysis on mismatched records. Once an activity is confirmed as being the same activity reported by different accreditors, they could perform an override and link the two records together. As a result there was no double counting. The mismatches and missing data were flagged appropriately.
One thing they did find were a lot of overlapping reporting dates. One submission may have multiple records and reporting periods that overlap on various records. We may need to provide additional information on reporting dates. Francis mentioned he and Valerie were talking about this yesterday. Valerie asked accreditors if that was problematic. Dimitra commented their providers didn’t have data so she couldn’t speak to that. Cynthia questioned how providers would know any problems that their data may have caused. Raj explained the confusion may arise due to different levels of data. The providers or vendors may have their own reporting data and it may not be on the same timeline.
Francis clarified the issue was accreditors are submitting data that has a different reporting period than providers. Linda commented that we were using an interim kind of system, and suggested that we need clarity in terms of what reporting period means. Francis asked if the confusion was around the report date or the date of the activity. Raj clarified that reporting start date and end date fields are required; activity date was optional. Linda commented the intent would be to simplify things for everyone, and to agree upon what the reporting period would be. So providers could plan ahead and accreditors would know when they were rolling it up. Valerie asked if she meant defining best practices. Linda mentioned if there was a logical time frame for having that reporting, we could align with that to simplify things. Raj thought that would help ease things because everybody knows the reporting period is standard. There would be an understanding that anything within that reporting period gets sent.
Valerie commented the challenge will be to find a logical place to align with. For universities it may be the academic year, other providers may use the calendar year. We don’t want to be prescriptive.
Linda asked how Boston University reports to PARS. Julie answered they typically report through the end of June. They can report more often, so it’s not a hardship. Joanna agreed. John commented whether they are collecting data in PARS using calendar or academic year, the system accommodates that. The report is based on that. Linda asked if additional activities that occur in the latter half of that year are reported. They try and assure there is sufficient time for providers to send data to accreditors and for accreditors to send to Polaris 60 days prior to July 9th (the date of the REMS). Adam, answered it is 60 days which is May 10th. Linda asked if there were any systems that would have difficulty working with that time frame.
Valerie noted the challenge is for providers not on the call who might have strong opinions about that. Linda suggested doing a follow-up on providers. July 9th is the date the FDA has to have the RPC’s completed report on their desk. The cut off is 60 days, and it has to have approval by various folks. Julie asked what that would mean for providers reporting on REMS activities; you would need to follow your accreditor’s time line to get into the system and still provide accreditors enough time. The RPC had 225 activities this past year. Each provider would report that to respective accreditors. Accreditors would need to roll-up multiple providers and they would have to send information to Polaris and have them look at flags that come up.
Valerie questioned whether PARS would be able to provide data according to those dates. John commented what your capturing is reporting year. It is difficult to create a report for the calendar year. It is something they need to discuss more internally at ACCME; PARS is capable of that. The completeness of the activity is an issue. Polaris receives information about activity multiple times.
Francis commented we should get buy in from providers that are not here. Linda added we need to make sure we hear from all accreditors. Valerie will create a survey and get it out to all providers receiving funding from the RPC, all known providers creating REMS training, and the accreditors. Linda suggested sending the survey to anyone on the working group as well. Valerie will take as an action item she and Francis will work on survey question draft.
Doris commented different groups have different reporting periods. The FDA is really more concerned about the first 10 months, data through May 10th. For next reporting period, it would be the entire year. Valerie noted the challenge is there are existing accreditation schedules that would be different; we would be asking them to go above and beyond what accreditors are requiring. Doris commented what they got from the last assessment were more activities than reported because it was the end of the reporting period. Linda commented that they want to report as much data by the May 10th date and yet be cognizant of the fact that providers and accreditors have existing systems and data to roll up, and we want to try and accommodate those as well. From an RPC standpoint, providers were terrific about getting data back. We could get an offline sense of things, what would happen between the cutoff and May 10 date so we don’t double report anything. Francis mentioned we will see what comes out of the survey; hopefully we will obtain good responses to guide our decisions. Doris stated the ultimate goal was to get as much data as possible to the FDA in the RPC report
3 Discussion of implementation guideline and necessity of 3.4.1
Valerie continued with the discussion of the implementation guidelines and the issues with position of prescribers, and the discussions with the ACCME. She directed the group to the table of contents page 16 in the REMS implementation guidelines where a number of different participation metrics are provided, the first one being the number of prescribers participating. In conversations with the ACCME, they mentioned that is not required by REMS and it’s not something they are including because they include prescribers successfully completing. Their suggestion was to take that out. Prescribers participating is not required, and it’s just confusing the issue. John thought that captured it. Cynthia commented she didn’t think it was there originally; it may have stemmed from discussion that a lot of folks participating and completing weren’t ER/LA Opioid prescribers, someone thought there was value to referring to number of “participants” instead of ERLA Opioid prescribers. Valerie thought maybe she put it in there in order to get to prescribers completing, first you have to talk about prescribers then prescribers successfully completing. Linda mentioned, from a REMS standpoint, the REMS document specifically refers to ER/LA Opioid prescribers, in the educational portion. Doris commented they have defined prescribers as those with a DEA license that have prescribed within the last year. Cynthia commented there would be value for CE providers to know about a nurse practitioner who might not have DEA registration but participated in a three hour educational intervention and assessment. Doris shared they had a discussion about this and it wouldn’t count for performance measures, it wouldn’t be required but they would leave it up to the providers. They would still need that vehicle to report it.
Valerie commented that it was a question of what do we want to provide guidance on. It would be possible and valid to use the standard to convey the number of prescribers participating or the number successfully completing whether or not those details exist in the implementation guidelines. Francis asked if the definitions would still be on the website. Valerie answered yes. Raj wanted clarification that is not part of the XML template and is not required. Valerie answered we only do prescribers successfully completing; however, if the FDA feels it is important to keep that section in optionally, we could do that. Linda asked the group if anyone had thoughts along those lines.
Cynthia thought that it is an important number; they try to reengage those who do not successfully complete to completion. For their purposes, it’s good to have a common definition, clarifying and stabilizing. Francis asked how big the number is of prescribers who do not complete. Cynthia commented it may be different for Julie and her teams, but they have about 9000 learners that have not yet taken the assessment. They are going to that group of candidates to push for completion.
Julie agreed it was beneficial to include them in tracking. Tracking that data has allowed them to obtain insights about their learners Valerie noted the document is not normative, we don’t have to adhere to everything in the document to be considered conforming to standard. It’s really for information purposes, so we can leave those sections in.
John commented that section 3.4 of the guidelines suggests that the FDA mandates the number of active ER/LA Opioid prescribers who participated. Valerie will revise the implementation guidelines to clarify what is mandated and send around the document; everyone can provide comments via email. John commented the language on 3.4.4 prescribers successfully completing, the word “prescriber” is misleading given how the FDA has defined it. Valerie asked if removing that would clarify. We could change schedule_2_or_3_registered_prescriber to schedule_2_or_3_registered_clinician and schedule_2_or_3_registered_prescribers_successfully_completing to schedule_2_or_3_registered_clinicians_successfully_completing. John thought that sounded like a reasonable change. Valerie asked the group if there were any objections to that. Valerie will make the revision and distribute. Francis asked if the polls will do that and are we ready to do this poll. Valerie suggested waiting on the poll.
4 Implementation progress
5 Draft Poll to send specification to standards committee
6 Conjoint committee meeting with the FDA; Dr. Kopelow's and Dr. Karty's testimonies at public FDA meeting
The conjoint committee met with the FDA and several government colleagues in July; Doris attended as well as many of her FDA colleagues. One outcome was an invitation to participate in a public meeting with the FDA. Dr. Kopelow and Dr. Karty’s participation and presentations were shared. This is positive news for the project, two very effective presentations. Julie commented that some folks shared information when they went back to their work; the dual challenge is just getting clinicians to recognize what REMS is and realize it’s important. The second challenge is getting folks to complete the full three hours plus assessments. It’s only been a few months since it started. Julie thanked Doris for hosting the meeting and thought it was a great opportunity to share information. The more we can talk to each other about it the more helpful it is. Doris thought it was a helpful meeting about blueprint issues. They hope to do it again in the next year. Valerie will create a survey and distribute it to the group; we may need another call to look at the results of the poll to come to a decision.
- We will create a draft survey of known REMS providers, accreditors, and the working group to see if providers are willing to report data on REMS activities to accreditors by a specific date and circulate for comment prior to sending out for response
- We will revise the implementation guidelines second sentence of 3.4 to indicate that collecting the number of prescriber participants is not mandated by the REMS.
- We will change the vocabulary term schedule_2_or_3_registered_prescriber to schedule_2_or_3_registered_clinician and update all references in the implementation guidelines
- We’ll redistribute the guidelines with the changes