Competences and Their Assessment

Moderated by Sean Hilton, M.D.

Effective and accurate competence assessment is at the core of health professions education, licensure, and certification efforts. And assessment methods and documentation conventions are constantly evolving.

Unconference Notes:

Questions for Consideration

1. Are these three broad domains comprehensive (Knowledge Skills Attitudes)?
2. Is competency more of a spectrum?
3. What are the sources of and barriers to competency data acquisition?

Audience Feedback

1. Are these three broad domains comprehensive (Knowledge Skills Attitudes)?

• KSA aren't themselves competencies but are necessary components. Competencies are a broader definition of a process of what doctors do, you need ksa. (KSA was presented as shorthand for broad domains of applied knowledge, clinical and technical skills; and behaviours and non-cognitive qualities required in combination for overall professional competence - hence the following bullet points)
• Missing demonstration of behavior.
• Add performance or outcomes to ksa .
• KSA are inadequate. Could be laid out in different taxonomies. Attitudes is another dimension.
  • There are micro to macro levels of competencies. Interprofessional competencies may be another level.
  • If you broke out on cognitive process and knowledge domain, you could lay out a framework for assessment. You could add additional dimension for attitudes. This is a modified Bloom's taxonomy.
• Has this led to a minimum standards, checkbox approach? A loss of aspiration to excellence? That is potentially an uncomfortable outcome of breaking down the holistic.
• There is a difference between what is a competency and what is competent. Competency defines what doctors need to do. When someone is competent, that is a measurement. What is the acceptable level of performance for the level of learner? The competency is the same.
  • Or is it more of a spectrum?At each level, there should be a safe practice threshold - ie manage cardiac disease in a safe way. Standards for cardiologists would be different than the standards for family medicine docs. It is not the same measure of competency. The assessment is different.
  • Is a competency a performance standard? Or a criteria?
• There is no consensus on taxonomies or dimensions. Yet all need to plan learning & assessment appropriate to competencies. We need to connect taxonomy to content and assessment in a way that can be shared. Declaring competence is another matter.
• As soon as we establish a standard of competence, it is outdated.
• How does competency change over time?
• It is a continuum

2. Is competency more of a spectrum?

• There is a spectrum - expectations need to follow professional development of the learner.
• Competence is made up of multiple competencies, described in multiple levels appropriate to the stage/level of physician.
• MedBiq has recognized interrelationsip of competencies. Competency description can detail the outcome. Emerging specs also recognize that learners have different levels of competency. There will be different expectations of different levels of learners for the same competencies.
• Practice processes are a combination of ksa. They reflect a spectrum from novice to expert.

3. What are the sources of and barriers to competency data acquisition?

• Sources of data - that's the big question. You need different approaches and tools for different competencies. We have some for communication and clinical skills. We need tools for professionalism, lifelong learning, reasoning. Expert judgement could be a way of assessing.
• Survey, self reports are sources.
• Is there bias inherent in the data when you know you are being measured? There are limitations to self assessment.
• Different types of assessment are necessary to measure different competencies. Objective measures of knowledge, reflections on score.
• We need a high stakes portfolio with referee.
• Nursing has a slightly different framework.
• Self assessment is required for Maintenance of Certification in the USA. The physician receives questions appropriate to scope of practice. Self assessment is also used for educational planning.
• Academy of Neurosurgery has an assessment; residents often take same assessment. But there are different expectations for different professionals. The case-based assessment is called SANS-Wired.
• Multidimensional approach requires multivariate measurement. How do you instruct and measure a set of actions? How do you validate empirically?

Participant Notes
The American Academy of Dermatology is renewing its scope of practice, including competencies (systems-based, ethics, etc). It starts with what general dermatologists need to know, then gets into subspecialties. At some point, you break - separate generalist competencies from specialist competencies. This core curriculum can then be handed to residency directors and medical schools to help shape their programs. In addition, they can give the competencies to physician assistant and nurse practitioner organizations so that they can determine levels of competence for specialists in those professions.

Add your ideas, presentations, or documents to this discussion!