Medbiq_logo_rtm.gif

 

 

 

 

 

MedBiquitous
Performance Frameworks

Research Report
 

January 17, 2013

 

 

 

 

 

Version: 0.1
Date: January 17, 2013
Authors: Laura Vail and Valerie Smothers
Author email: vsmothers@jhmi.edu


Contents

1 Introduction

2 Organizations Researched

3 Findings

3.1 Existing Standards and Specifications

3.2 Common Education Data Standards

3.3 Ed-Fi

3.4 CEN Workshop Agreement 15455 "A European Model for Learner Competencies"

3.5 e-Competence Frameworks

3.6 Europass CV/Language Passport v2.0

3.7 HR-XML

3.8 InLOC

3.9 ISO Model for Competency: Information Technology for Learning, Education and Training- Information Model for Competency

4 Best Practices

 


MedBiquitous Consortium XML Public License and Terms of Use

 

MedBiquitous XML (including schemas, specifications, sample documents, Web services description files, and related items) is provided by the copyright holders under the following license. By obtaining, using, and or copying this work, you (the licensee) agree that you have read, understood, and will comply with the following terms and conditions.

The Consortium hereby grants a perpetual, non-exclusive, non-transferable, license to copy, use, display, perform, modify, make derivative works of, and develop the MedBiquitous XML for any use and without any fee or royalty, provided that you include the following on ALL copies of the MedBiquitous XML or portions thereof, including modifications, that you make.

  1. Any pre-existing intellectual property disclaimers, notices, or terms and conditions. If none exist, the following notice should be used: “Copyright © [date of XML release] MedBiquitous Consortium. All Rights Reserved. http://www.medbiq.org”
  2. Notice of any changes or modification to the MedBiquitous XML files.
  3. Notice that any user is bound by the terms of this license and reference to the full text of this license in a location viewable to users of the redistributed or derivative work.

 

In the event that the licensee modifies any part of the MedBiquitous XML, it will not then represent to the public, through any act or omission, that the resulting modification is an official specification of the MedBiquitous Consortium unless and until such modification is officially adopted.

THE CONSORTIUM MAKES NO WARRANTIES OR REPRESENTATIONS, EXPRESS OR IMPLIED, WITH RESPECT TO ANY COMPUTER CODE, INCLUDING SCHEMAS, SPECIFICATIONS, SAMPLE DOCUMENTS, WEB SERVICES DESCRIPTION FILES, AND RELATED ITEMS. WITHOUT LIMITING THE FOREGOING, THE CONSORTIUM DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY, EXPRESS OR IMPLIED, AGAINST INFRINGEMENT BY THE MEDBIQUITOUS XML OF ANY THIRD PARTY PATENTS, TRADEMARKS, COPYRIGHTS OR OTHER RIGHTS.  THE LICENSEE AGREES THAT ALL COMPUTER CODES OR RELATED ITEMS PROVIDED SHALL BE ACCEPTED BY LICENSEE “AS IS”.  THUS, THE ENTIRE RISK OF NON-PERFORMANCE OF THE MEDBIQUITOUS XML RESTS WITH THE LICENSEE WHO SHALL BEAR ALL COSTS OF ANY SERVICE, REPAIR OR CORRECTION.

IN NO EVENT SHALL THE CONSORTIUM OR ITS MEMBERS BE LIABLE TO THE LICENSEE OR ANY OTHER USER FOR DAMAGES OF ANY NATURE, INCLUDING, WITHOUT LIMITATION, ANY GENERAL, DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, OR SPECIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF ANY USE OF MEDBIQUITOUS XML. 

LICENSEE SHALL INDEMNIFY THE CONSORTIUM AND EACH OF ITS MEMBERS FROM ANY LOSS, CLAIM, DAMAGE OR LIABILITY (INCLUDING, WITHOUT LIMITATION, PAYMENT OF ATTORNEYS’ FEES AND COURT COSTS) ARISING OUT OF MODIFICATION OR USE OF THE MEDBIQUITOUS XML OR ANY RELATED CONTENT OR MATERIAL BY LICENSEE.

LICENSEE SHALL NOT OBTAIN OR ATTEMPT TO OBTAIN ANY PATENTS, COPYRIGHTS OR OTHER PROPRIETARY RIGHTS WITH RESPECT TO THE MEDBIQUITOUS XML.

THIS LICENSE SHALL TERMINATE AUTOMATICALLY IF LICENSEE VIOLATES ANY OF ITS TERMS AND CONDITIONS.

The name and trademarks of the MedBiquitous Consortium and its members may NOT be used in advertising or publicity pertaining to MedBiquitous XML without specific, prior written permission. Title to copyright in MedBiquitous XML and any associated documentation will at all times remain with the copyright holders.

 

 


1          Introduction

As part of the standards development process within the Competencies Working Group, MedBiquitous has researched existing standards and specifications that could be assets in the development of a performance framework specification. As a matter of best practice, MedBiquitous build on existing standards when it is practical to do so. MedBiquitous staff researched credible learning technology standards developers and relevant open tools. A summary of the findings for each standard or specification researched is included along with information on relevant vocabularies, best practices, and recommendations.

 

2          Organizations Researched

The following organizations were researched:

  • BSI Group
  • The European Committee for Standardization (CEN Workshop Agreements 14927 and 15455)
  • European e-Competence Framework (Framework 2.0)
  • IMS Global Learning Consortium (ePortfolio)
  • Ed-Fi Alliance
  • Europass (Certificate Supplement and Language Passport)
  • HR-XML (ProficiencyLevel)
  • InLOC
  • International Organization for Standardization (Information Technology for Learning, Education and Training)
  • National Center for Education Statistics (US)

 

Within each organization, a search for performance level or competency level related standards, specifications, and systems was conducted.

3          Findings

CEDS and ED-Fi offer ways of communicating performance levels, but Performance levels are defined as ranges of assessment scores with no inherent connection to a competency delineated. While HR-XML provides proficiency levels, they also are expressed as text or a numeric score with no connection to a larger framework.

Europass related specifications, while providing a clear way to indicate learner levels of language skills based on the Common European Framework of Reference on Languages, does not provide a common framework for describing levels of performance. IMS descriptions of levels are free text fields for describing levels very generally (ie Entry, Honors). The XML for the Language Passport, where levels are most applicable, provides only a free text description of the skill level associated with foreign language competence. Other aspects of the European move to make competency transparent, such as the e-Competences for ICT professionals, lack a technical binding. InLOC provides a highly flexible, if abstract, mechanism for creating performance level frameworks, but its development is still very much in progress.  The abstract nature of the data model makes it more difficult to implement for simpler implementations.

The ISO specifications are promising, but they are still under ballot and have no binding provided. The complexity of the data model may make it more difficult to implement for simpler frameworks.

If the MedBiquitous Competencies Working Group is to leverage any of these specifications, we must take into account the following:

  • Any licensing or Intellectual Property policies of the parent organization and their approach to derivitative works made by external organizations.
  • The ability to extend the specification. None of the works cited address the US continuing education or residency environments, so a mechanism for extension suitable to including that data would be necessary.
  • Technical constraints and tradeoffs. In some cases very general approaches to representation are used. Overgeneralization can make adaptation to a particular domain difficult if the mappings to that domain are not clear to implementers. Or sometimes older technologies are used where newer technologies are available.
  • Adoption. If an existing specification has broad adoption and is highly applicable, leveraging that specification would increase the scope of interoperability and increase chances for broader adoption.  

3.1       Existing Standards and Specifications

 

3.2       Common Education Data Standards

https://ceds.ed.gov/

The goal of CEDS is to enable nationwide educational institutions from early learning through postsecondary to understand, compare and exchange data in an accurate, timely, and consistent manner ( https://ceds.ed.gov/whatIsCEDS.aspx ). Use of the standard would make it easier for states to track individual students as they move between institutions and grade levels. The CEDS project is coordinated by National Center for Education Statistics, and participation is voluntary.

Several Assessment Elements address levels ( https://ceds.ed.gov/da t a/xls/ceds-rtta-elements-version3.xls ):

  • Assessment Performance Level Identifier (line 12): Unique number or alphanumeric code assigned to an assessment performance level
  • Assessment Performance Level Label (line 13): Label representing the performance level appropriate for use on a report
  • Assessment Performance Level Lower Cut Score (line 14): lowest possible score for the performance level
  • Assessment Performance Level Score Metric (line 15): The metric or scale used for score reporting. (Five-digit codes are assigned for metrics such as ACT score, Letter grade/mark, and Percentile.)
  • Assessment Performance Level Upper Cut Score (line 16): Highest possible score for the performance level
  • Assessment Performance Level Name (line 178): The level of performance of students on an assessment. (Level0-Level6; Level 0 is used for students who do not participate, not used by states that do not assign nonparticipants a level. Level 1 is the lowest level assigned to students who completed the assessment and obtained a valid score.)
  • Performance Level Descriptive Feedback (line 187): Diagnostic statements or other feedback to the student or teacher specific to the performance level


Initial lists of elements to include and definitions were drawn from NCES Handbooks; Schools Interoperability Framework (SIF); Postsecondary Electronic Standards Council (PESC); Education Data Exchange Network (EDEN); the Integrated Postsecondary Education Data System (IPEDS); National Forum on Education Statistics; Head Start; other federal and national data sources; as well as best practices from states, districts, and institutions of higher education ( https://ceds.ed.gov/FAQ.aspx ).

CEDS uses an XML schema ( https://ceds.ed.gov/FAQ.aspx ) which "supports standardizing educational organizations and their relationships with other organizations, people and time" ( https://ceds.ed.gov/dataModel.aspx ). It includes a web-based Alignment tool ( http://ceds.ed.gov/alignmenttool ) which allows users to compare their institution's data dictionary to both   other institutions' data dictionaries and to CEDS.

 

3.3       Ed-Fi

http://www.ed- f i.org/

Ed-Fi "extracts student information from a variety of educational data systems, and then standardizes, integrates and communicates it to educators and other parties through Web-based dashboards, reports and other applications" ( http://www.ed-fi.org/faqs/#what-is-ed-fi ). It is vendor-neutral, XML-based and designed to integrate information from a range of existing sources. It can be used by educational organizations to meet federal reporting requirements. It was developed for the K-12 sector, and participation is voluntary.

It is made of the following components ( http://www.ed-fi.org/faqs/#from-a-technical-perspective-what-is-ed-fi-and-what-does-it-include ):

  • Unifying Data Model: The central XML schema that defines the common data elements on which the standard is based
  • Data Exchange Network: Includes the interchange schema built on the core schema
  • Application Framework: Includes a technical implementation guide that provides prescriptive guidance and general concepts essential to creating Ed-Fi based applications
  • Dashboard Source Code: Provides sample dashboard elements and educational metrics that educators have identified as important

It is based on and aligns with the U.S. Department of Education's Common Education Data Standards (CEDS) ( http://www.ed-fi.org/faqs/#what-is-ed-fi ). With respect to performance levels, the XML schema includes the following elements ( http://www.ed-fi.org/technical-documentation/ ):

  • AssessmentScore: Definition of the scores to be expected from this assessment
  • AssessmentPerformanceLevel: Definition of the performance levels and the associated cut scores. Three styles are supported: 1. Specification of performance level by minimum and maximum score 2. Specification of performance level by cut score, using only minimum score 2. Specification of performance level without any mapping to scores"
  • FinalLetterGradeEarned: The final indicator of student performance in a class as submitted by the instructor
  • FinalNumericGradeEarned: The final indicator of student performance in a class as submitted by the instructor
  • PerformanceLevel: The performance level(s) achieved for the assessment

 

3.4       CEN Workshop Agreement 15455 "A European Model for Learner Competencies"

ftp://ftp.cenorm.be/PUBLIC/CWAs/e-Europe/WS-LT/CWA15455-00-2005-Nov.pdf

This document addresses how the IMS Reusable Definition of Competency or Educational Objective (RDCEO) specification and IMS Learner Information Package (LIP) can be used to communicate standardized learning competency data. It aims to have interoperability between information models for CEN Competency Definitions, Competency Records, and Job Definitions (p 9). The CWA recommends extending the IMS LIP Competency data structure “to provide a placeholder for expression of level of performance for a specific competency.” The document then shows how the Europass Framework for Transparency of qualifications & competencies maps to the proposed data structure. This is significant in that the European Language Portfolio provides a way to report levels of competence with regard to specific language skills.

The Common European Framework of Reference on Languages specifies these levels.  A part of the Level C1 (Effective Operational Proficiency or Advanced) is Listening, which is described as "Can understand extended speech even when it is not clearly structured and when relationships are only implied and not signaled explicitly. Can understand television programs and films without too much effort" (p14). Specific performance indicators are provides that provide more detailed examples of that skill level.

The CWA recommends describing these as Complex Competency Definitions, ie competencies that require other competencies. The examples do not show the relationships among different levels of the same skill (ie European B1 Spoken Language Skill and European C1 Spoken Language Skill). It illustrates the C1 (Effective Operational Proficiency or advanced) Language Skill level, which is composed of five basic language skills that are primitive competency definitions: Listening (described above), Reading, Spoken Interaction, Speaking, and Writing (p16). The C1 level is a complex competency object composed of primitive RDCEO instances created for each of these elements. Its description is

The specification uses RDCEO and an XML binding. It became Workshop Agreement Status November 2005, though it is unclear whether the recommendation was adopted formally in other places.

Standards it builds on include: ISO JT1/SC36 – WG3 “Participant Information” by the French Standardization Body, which “defines a competency-related set of terms and concepts and establishes the relations which exist between the concepts and between the concepts and the terms defined” (p 7).

3.5       e-Competence Frameworks

http://www.ecom p etences.eu/

This framework is intended for use by ICT Professionals in diverse roles across industries ( http://www.ecompetences.eu/1386,Home.html). It is used to describe the levels of the competencies of ICT systems/organizations, rather than individuals. "The European e-Competence Framework 2.0 provides a basic, clear and sound orientation for companies who need to take decisions about recruitment, career paths, training, assessment, etc. It is also useful for promoting clearer understanding of company competence needs" ( http://www.ecompetences.eu/1855,Framework+overview.html ).

It identifies 36 e-Competences (such as Design Architecture and Application Design) within five e-Competence Areas in ICT (Plan, Build, Run, Enable, and Manage). Each of the e-Competences can be ranked as one of five levels (e-1 to e-5) (http://www.ecompetences.eu/1855,Framework+overview.html ), with the higher numbers signifying higher competence. The ranking system correlates with European Qualifications Framework (EQF) levels 3-8 used by the European Commission for Education and Culture ( http://www.ecompetences.eu/site/objects/download/4550_EQFbroch2008en.pdf ). In that scale, Level 1 includes school leaving certificate while Level 8 includes Doctorates (“The European Qualifications Framework for Lifelong Learning,” p3).

EQF includes ranking scales for both Knowledge and Skills (full descriptions follow), and the e-Competence labels its e-1 to e-5 ranking as "proficiency levels," and does not specify whether it correlates with Knowledge of Skills .

The e-Competence Frameworks was approved as a CEN Workshop Agreement since September 2010. The EQF was approved by the European Parliament on April 23 2008. It is a specification only, and does not have a binding. There is an online tool that allows users to create HTML documents specifying their competence.

3.6       Europass CV/Language Passport v2.0

http://europass.cedefop.europa.eu/ e n/home

The free online tools can be used to create five standardized documents on EU residents seeking jobs: individuals can create for themselves a CV and Language Passport, and education and training organizations can create one fo the following: Europass Mobility (on skills acquired in another country), Certificate Statement (skills from vocational and training certificates) and Diploma Supplement (higher education) ( http://europass.cedefop.europa.eu/en/about ).

It’s objectives are:

  • to help citizens communicate their skills and qualifications effectively when looking for a job or training;
  • to help employers understand the skills and qualifications of the workforce;
  • to help education and training authorities define and communicate the content of curricula ( http://europass.cedefop.europa.eu/en/about )

Certificate Supplement

Diploma supplements have levels that correspond to levels of education but no competency or proficiency levels.

Language Assessment (component of Language Passport)

The "Language Self Assessment Grid" ( http://europass.cedefop.europa.eu/en/resources/european-language-levels-cefr ) is most related to performance levels, and is a grid on which individuals can rank their language skills in the categories Understanding (subsections Listening and Reading), Speaking (Spoken Interaction and Spoken Production) and Writing. There are six possible levels, which in increasing order of ability are A1, A2, B1, B2, C1, and C2. From http://en.wikipedia.org/wiki/Common_European_Framework_of_Reference_for_Languages :

  • Level A: Basic User
    • A1: Breakthrough or beginner
      • Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type.
      • Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has.
      • Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.
    • A2: Waystage or elementary
      • Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very basic personal and family information, shopping, local geography, employment).
      • Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters.
      • Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.
  • Level B: Independent User
    • B1: Threshold or intermediate
      • Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc.
      • Can deal with most situations likely to arise whilst traveling in an area where the language is spoken.
      • Can produce simple connected text on topics which are familiar or of personal interest.
      • Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans.
    • B2: Vantage or upper intermediate
      • Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialization.
      • Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party.
      • Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options.
  • Level C: Proficient User
    • C1: Effective Operational Proficiency or advanced
      • Can understand a wide range of demanding, longer texts, and recognize implicit meaning.
      • Can express him/herself fluently and spontaneously without much obvious searching for expressions.
      • Can use language flexibly and effectively for social, academic and professional purposes.
      • Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organizational patterns, connectors and cohesive devices.  
    • C2:   Mastery or proficiency
      • Can understand with ease virtually everything heard or read.
      • Can summarize information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation.
      • Can express him/herself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in the most complex situations.

The descriptions for the Understanding: Listening proficiency ( http://europass.cedefop.europa.eu/en/resources/european-language-levels-cefr ) are:

  • Level A
    • A1: I can understand familiar words and very basic phrases concerning myself, my family and immediate concrete surroundings when people speak slowly and clearly.
    • A2: I can understand phrases and the highest frequency vocabulary related to areas of most immediate personal relevance (e.g. very basic personal and family information, shopping, local area, employment). I can catch the main point in short, clear, simple messages and announcements.
  • Level B
    • B1: I can understand the main points of clear standard speech on familiar matters regularly encountered in work, school, leisure, etc. I can understand the main point of many radio or TV programs on current affairs or topics of personal or professional interest when the delivery is relatively slow and clear.
    • B2: I can understand extended speech and lectures and follow even complex lines of argument provided the topic is reasonably familiar. I can understand most TV news and current affairs programs. I can understand the majority of films in standard dialect.
  • Level C
    • C1: I can understand extended speech even when it is not clearly structured and when relationships are only implied and not signaled explicitly. I can understand television programs and films without too much effort.
    • C2: I have no difficulty in understanding any kind of spoken language, whether live or broadcast, even when delivered at fast native speed, provided. I have sometime to get familiar with the accent.

Schemas for JSON and XML are available for this specification ( http://europass.cedefop.eu r opa.eu/en/resources/for-developers ). The XML schema provides free text elements for indicating the level associated with listening, reading, spoken interaction, spoken production, and writing.

It was adopted as the single framework for transparency of qualifications and competencies by the European Parliament in Dec 2004 ( http://europass.cedefop.europa.eu/en/about/history ). Since the launch of its website in May 2005, there have been nearly 1,500,000 total visits ( http://europass.cedefop.europa.eu/en/resources/statistics ).

3.7       HR-XML

http://www.hr-xml.org/

This nonprofit describes its purpose as, "Dedicated to the development and promotion of a standard suite of XML specifications to enable e-business and the automation of human resources-related data exchanges" ( http://www.hr-xml.org/?page=About ). Members can download the full release of its XML specifications or those specific to Assessment, Benefits, Employee Performance Management, etc. Membership and the specifications are free.

The specifications provide a way to track Assessments and Performance Management information. A diagram of the relationship between its specifications is at https://hr - xml.site-ym.com/?page=DownloadLandingPage .

The most relevant part of its documentation to Performance levels is the ProficiencyLevel element, and the XML documentation follows:

  • ProficiencyLevel definition: "The state or degree of mastery or adeptness obtained or achieved for a given competency. A proficiency level is expressed as a score a point scale or a mark among range of values."
  • Types (all are complexType):
    • BaseScore: either ScoreNumeric, ScoreText, or UserArea. All in range 0-unbounded
    • ScoreNumeric: "A numerical record of the marks allotted to individuals in the measurement of abilities, capacities to learn, in the assessment of personality, or in other measurable characteristics (e.g., credit worthiness). Includes a set of attributes to identify the scoring scheme and its characteristics."
      • Attribute: scoreNumericCode "A code used to numerically classify the type scoring methodology or convention."
      • Attributes: minimumScoreNumeric and maximumScoreNumeric, both are optional
    • ScoreTextType: "A non-numeric or textual representation of a Score. A record of the marks allotted to an individual in the measurement of abilities, capacity to learn, in the assessment of personality, or in other measurable characteristics (e.g., credit worthiness). Represents either the total number of points awarded, or a general standard achieved. Includes a set of attributes to identify the scoring schemes."
      • Attribute: ScoreTextCode "A code used to textually classify the type scoring methodology or convention"
      • Attributes: minimumScoreNumeric and maximumScoreNumeric, both are optional
      • UserAreaType: "Allows the implementer to extend the base HR-XML specification. This is done by defining the additional information in XML Schema and referencing the new schema in the xml instance document through the use of namespaces. Once this is done the additional information defined there can be carried in the BOD XML instance document." (HR-XML Consortium, 3.2 Assessments Specifications. October 31, 2012. http://www.hr-xml.org/?page=DownloadLandingPage )

Use of the standard is voluntary only, and its most recent full release is Version 3.2.

3.8       InLOC

http://wiki.teria.no/display/inloc/Home

InLOC ("Integrating Learning Outcomes and Competences") is a model for information on learning outcomes and competences (LOCs). It can be used for personal, professional and vocational development, human resources and employee performance management, training and education, whether in the workplace or in school, vocational or higher education ( http://wiki.teria.no/display/inloc/Introduction+to+the+Model ).

It describes the goal of its framework as follows: "InLOC does not set out any specific LOC concept definitions or structures for any application area, but rather, it provides a common format in which those definitions and structures may be represented" ( http://wiki.teria.no/display/inloc/Scope+of+the+Model ).

While there are diverse ways of representing levels, "the common ground is that in each case, there is the idea of possible progression, with higher "levels" (or other term) coming after, and subsuming, lower ones. The assumption is always that someone at a higher level is also able to do things that are able to be done by someone at a lower level of the same competence" ( http://wiki.teria.no/display/inloc/InLOC+treatment+of+levels ).

InLOC’s data model uses the concept of triples, similar to the Subject Verb Object structure of sentences. So the following statement:

The competency History taking has performance level Collects comprehensive history without prioritization .

May be represented.

Defining Levels in InLOC :

  1. The definitive description of each level is represented as a LOCdefinition .
  2. What these are levels of is also represented as a LOCdefinition .
  3. The LOCstructure for the level framework includes LOCassociation s that

The number is often taken directly from the level framework documentation, but not always. Occasionally, a level system uses non-numeric level values, or uses numeric values where a higher level value means a lower level of competence. To enable automatic comparison of levels, it is essential that all level numbers work in the same sense. Any InLOC representation of level definitions will ensure that each level is given a number in the normal sense, i.e. where a greater number represents a higher level of competence, or an intended learning outcome that subsumes a lower level one.

(http://wiki.teria.no/display/inloc/InLOC+treatment+of+levels )

Attributing Levels in InLOC

In many competence frameworks, levels are not defined in the sense above, but rather levels from a different framework, already existing, are attributed to the LOCdefinition s of the competence framework. This can be represented simply in InLOC using the level type of LOCassociation .

  1. Create a LOCassociation within the LOCstructure .
  2. The type of the LOCassociation is level .
  3. The subject is the LOCdefinition having a level attributed to it.
  4. The scheme is the level scheme containing the level definition.
  5. The object is the attributed level:
    • the obj e ct id may be the identifier of the particular level, if there is an identifier. InLOC recommends URIs for identifiers.
    • the object label may be the term used for the particular level – this is particularly useful for non-numeric levels.
  6. The number is the attributed level number.

The attributed level number would often be a level number that has been explicitly defined in the level scheme, but it is not necessarily so. It is perfectly possible to attribute a level number that is between defined numbers, though the meaning of this may not be well-defined. For instance, one could attribute an EQF level 4.5 to something that appeared to fall between the definition of EQF level 4 and of EQF level 5.

(http://wiki.teria.no/display/inloc/InLOC+treatment+of+levels )

InLOC’s framework can use XML, RDF and JSON for bindings ( http://wiki.teria.no/display/inloc/Bindings) . It is currently preparing for consideration to be a CEN workshop agreement, with the eventual goal of becoming a European Standard.

Other frameworks it uses or builds on are Dublin Core Metadata Initiative's DCMI Metadata Terms and W3C's Semantic Web work on SKOS . Other structures that define levels that were examined during its creation are: Common European Framework of Reference for Languages, Australian Core Skills Framework, e-Competence Framework, Vitae Researcher Development Framework ( http://wiki.teria.no/display/inloc/InLOC+treatment+of+levels ).

3.9       ISO Model for Competency: Information Technology for Learning, Education and Training- Information Model for Competency

Part 1:   ISO/IEC 20006-1, Competency General Framework and Information Model

Link to committee: http://isotc.iso.org/livelink/livelink/open/jtc1sc36

It describes its purpose as:

  • A general framework for dealing with competency information in information technology (IT) used for learning, education, and training (LET) contexts;
  • A system architecture for managing and exchanging competency information and its related objects.
  • An information model for expressing competency and its related objects that includes an introduction to the composition of
    • Basic competency information ("main elements may include “identifier”, “name”, “description” and so on, which attributions and elements are setup in RDCEO or HR-XML competencies. These are simple and essential for the exchange of competency information" (p 14));
    • Semantic competency information ("may be used to express the content of competency, which includes meaning and context of competency" (p 14));
    • Supplemental competency information ("may be used to modify or arrange a certain competency in order to adapt an individual institutional or organization usages" (p 14)).
  • Use cases used to support the development of the general framework and competency information model (p 1).

ISO identifies 6 entities that are parts of competency architecture, illustrated in an architecture diagram (p 6-7)

  1. e-Profile : a set of records that pertain to an individual, eg personnel records, student information system records on an individual
  2. Competency information : it can be expressed in a variety of ways, including natural language and taxonomic structures (eg national occupational classification). Systems may use different notation depending on the type of expression used (though that can be a challenge for interoperability, which RDCEO and RCD aim to address)
  3. Competency semantic information : "represents compliant instantiated data for semantics of competency contents and deals with the relationships among different competency structures" (p 7). Necessary because "Competency Information" cannot deal with semantic information.
  4. Level information : "Level information compensates for the lack of information on competency or learning objectives. It is hard to define learning objectives without information regarding level. Also, human and educational assessments are executed referring to level information. Though the level information can be integrated into competency information itself, it is more effective to define and to manage level information and competency information separately for interoperability" (p 7). (The meaning of this is not clear to me.)
  5. Evidence information : supports the validity and reliability of the information in the e-Profile. Eg "grade: A" in the e-Profile could be supported in this section by an online learner's activity logs, learning results, exam scores, observation, etc.
  6. Assessment method and method information : refers to the ways, processes, and rules of assessment that are used to acquire the competency information; it is important to make the elements of assessment clear. What is measured by a different method is generally considered to be a different competency.

The description of levels is flexible and can vary depending on the needs of the organization and what is to be expressed.

There are five structural relation patterns (p 9-10)

  1. General-specific : parent competency reflects a categorical concept or name of child competencies. Eg for an automobile, parent is automobile and children are bus, truck, family car, etc, where all automobiles share an essential function "to move fast transporting humans or others"
  2. Whole-partial : children are parts or elements of a parent. Eg for parent competency 'social skill,' children are social awareness skill and social understanding skill, which are related to each other and are part of parent competency
  3. Universal-particular : parent and children share same meaning or character but are applied in different contexts. Eg parent automobile, children sporty car, sedan, wagon, etc. Or parent social skill, children social skill in school and social skill in family
  4. Abstract-concrete : children are instances of the parent eg parent automobile, children are cars from specific builders
  5. Level : used to figure out degree of competency. Parent can be further described using child relations that are used to express the degree or grade; children can express the degree of execution.

It is an information model rather than having a binding itself; it recommends throughout using RDCEO and RCD.

Its status is as a working draft for ballot or comment, and a vote was expected in 9/2012. On 10/30/12, on the committee's site (URL above) there are new documents for 20006-1 and 20006-2 on the "disposition of comments... resulting from the CRM held 2012-09-13" ie Sept 13 (documents modified 10/11/2012). The documents' status described as "FYI," and list committee members' comments, suggestions for modification, and whether the modifications were accepted. So, appears no final vote yet.

Other standards that it built on are ISO/IEC TR 2 4763, ISO/IEC 20006, ISO/IEC TR 24763.

ISO Part 2: ISO/IEC 20006-2, Proficiency Level Information Model (IMC-P)

This ISO candidate standard provides an information model for data about competency proficiency and level. They have a notion of proficiency that is "progress, advancement or improvement in a competency." The information model includes proficiency levels that describe degree or grade of competency or proficiency. Building on these concepts is the concept of proficiency level sequence, "a value set of proficiency level," including nominal steps, numbered steps, interval scale, ratio, etc. Proficiency levels may be represented in a decreasing pattern or increasing pattern (level 1 to level 10), a yes/no binary (competent or incompetent), "term patterns" where the levels are named (bachelor's, master's, etc), or alternating patterns (in Judo you progress form Kyu 10 through 1 with 1 being the higher level, then to Dan 1 through 10, with 10 being the highest level), and parallel patterns, where two different proficiency levels exist in parallel (as in skiing, where 1st rank learners may progress to different instructor levels or to different technical certification levels). A proficiency composition model allows one to create multiple proficiency sequences for the more complex models like judo or skiing.

Proficiency information is broken into levels and dimensions. Dimensions are the factors, assessment elements, or criteria. Dimensions define the differences in proficiency levels and may be used in assessment. The standard requires that you specify the number of levels in the schema (ie if there are 5 levels, you indicate that), or if the range is continuous (1-990), continuous may be indicated. For each level, the position in relation to other levels must be defined (ie level 1 of 5, level 2 of 5, etc). Dimensions may have more specific criteria.

It is the second part of the multi-part standard, which provides an overarching information model for competency data. "This standard will provide a reference code and protocol to manage and exchange information about knowledge, skills, ability, attitude, or educational objectives" (p 5).

Proficiencies and levels can all have simple (integers) and complex (has attributes such as id, name, and description and possibly other more complex items (such as id, name, and description), properties, and elements) types.

There are five general types in the proficiency composition information model:

  1. Increasing integers in primary school grades (2nd, 3rd grades);  
  2. Names for progression of university degrees (bachelors, masters);
  3. Yes/no (professional certification exam);
  4. Midterm switching patterns (the grouping of grades into primary, secondary school);
  5.   Complex types for judo and Japanese skiing.

It is an information model only, and does not have a binding

Like the first part of the standard above, it is text for ballot or comment rather than an approved standard.

The   information model is very flexible and highly detailed. The flexibility and level of detail also make it difficult to follow for simpler implementations. It's unclear as the use case for the level of specificity in some cases; it may be more applicable in highly structured assessment environments as are in place in Japanese higher education.

It describes how it can be applied to the European Qualifications Framework (EQF) and Japanese National Skills Standard (ITSS)

4          Best Practices

Use of URIs as identifiers is a best practice identified in at least one of the standards researched. this is also viewed as a best practice or requirement in other MedBiquitous specifications.