24 responses were received for the survey.
75% of organizations surveyed are already delivering competency-based learning or planning to do so. Numerous barriers to implementation were cited, the most frequent being:
- the difficulty in evaluating/assessing competencies in a scalable fashion,
- faculty and staff time required for creation and implementation of a competency-based system,
- lack of faculty and staff expertise in competency-based education and constrained staff resources for developing a competency-based system,
- the challenge of curriculum reform,
- lack of clear, evidence-based competency definitions to work with, and
- a lack of software support for the goals for competency-based learning.
Respondents reported that they would consider using technology standards to support competency-based education and assessment for a broad range of contexts:
- Medical education (75%), nursing (38%) and allied health (33%)
- Undergrad (38%), postgrad (46%) and CPD (54%)
- Maintenance of competence (33%) and certifcation/licensure (33%)
- 12.5% reported that they would not consider using technology standards at this time.
In ranking the different uses of technology standards (based on our working group use cases) on a 7-point scale with 1 being not important and 7 being important, respondents ranked all of the uses cited as being somewhat important. In descending order of importance, the uses are:
- Record performance data as evidence of competence for learning or assessment (mean 6.0)
- Search a system to find curricular components (for example learning activities or assessments) addressing a particular competency (5.8)
- Record activities related to and evidence of competence within a portfolio that details the expected competencies and how the individual has progressed relative to each competency (5.8)
- Search a system to develop a report that details what competencies are addressed in the curriculum (5.7)
- Direct a clinician to learning based on gaps in competence (remediation) (5.5)
- Qualify the link between a competency/ontology and a person or curricular component, (e.g. the degree to which a learner has demonstrated a competency) (5.5)
- Index a competency framework against a controlled vocabulary, terminology, or ontology (5.4)
- Compare competence data for a clinician against a benchmark (gap analysis) (5.3)
- Export a portfolio or portion of a portfolio (5.3)
- Map one set of competencies to another (5.1)
- Use a machine readable set of competencies or learning outcomes developed by a collaboration or another organization (import it into the system) (4.6)
- One additional use of competency standards offered was that of federated search strategies for finding articles, resources or activities linked to competencies.
When asked if they use or plan to use a technology platform to accomplish certain tasks, the majority of respondents indicated that they currently use technology platforms to develop and deliver learning materials. The majority indicated that they currently or plan to implement multiple sets of competencies for a single curriculum, and implement controlled vocabularies, taxonomies, ontologies to support competency-based learning. The majority are planning to use technology platforms to associate curricular components with competencies, track learner progress towards competency, track where competencies are addressed in the curriculum, and recommend learning based on gaps in competence. The ability to export a competency-based portfolio was equivocal.
When asked what a competency definition should include, 95% indicate that it should include a competency statement. A majority also indicated that it should include narrative description of the competency, typing (knowledge, skill, attitude, role, etc.), conditions of performance, performance criteria, outcome criteria, and recommended evaluation and assessment methods. Only 26% wished to represent multiple languages within a framework. One respondent added the ability to define different levels of achievement based on the level of the learner (beginner, intermediate, advanced).
When asked what a competency framework should include, 100% of respondents indicatd that it should include groupings of competencies under a category, role, or other structure. 90% indicated that a framework should describe hierarchical relations. A majority indicated that it should also include non-hierarchical relationships and narrative descriptions/introductions for categories or other components. A minority wished to include references/associations to outside resources. One respondent added links to an evidence base(related to references).
When asked if they would find technology standards useful on a scale of 1 to 7 with 1 being not useful and 7 being useful, 78.3% of respondents indicated that they would find technology standards useful (7) or somewhat useful (6).