As part of the standards development process within the Educational Achievement project, MedBiquitous has researched existing standards and specifications that could be assets in the development of an educational achievement specification. As a matter of best practice, MedBiquitous builds on existing standards when it is practical to do so. MedBiquitous staff researched credible learning technology standards developers and relevant open tools. A summary of the findings for each standard or specification researched is included along with information on relevant vocabularies, best practices, and recommendations on the use of competencies. Comments and suggestions for additional research are welcome.
The following organizations were researched:
- CEN (The European Committee for Standardization) ( complete )
- HR-XML (A consortium for interoperability in human resources) ( in progress )
- IEEE Learning Technologies Standards Committee (IEEE LTSC)
- IMS Global Learning ( complete )
- The UK Joint Information Systems Committee (JISC)
- The Post Secondary Education Standards Council (PESC) ( complete )
- Schools Interoperability Framework (SIF)
Within each organization, a search for assessment and competency related standards, specifications, and systems was conducted.
Europass XML Schema v 2.0
Europass is a European Union directive that seeks to make skills and qualifications easily understood across Europe. It includes tools, templates, and specifications for cvs and language skill assessment called a Language Passport. Europass includes identifying information about the individual, their occupation, work experience, education, language, and skills.
European Learner Mobility Achievement Information (EuroLMAI)
“The EuroLM Achievement Information Model seeks to address directly the European requirement for rapid implementation and dissemination of the institution-owned transparency information tools throughout Europe. This EuroLM Achievement Information (EuroLMAI) Model will be developed as a generic model for representing the institution attested achievements of a learner within a formal learning setting, resulting from the learner's participation in one or more learning opportunity instances and including any available associated results. The Europass institution-owned learner achievement information structures, namely the Europass Diploma Supplement and the Europass Mobility, as well as other official achievement reports (e.g. the learner's Transcript of records), will be specified as application profiles of the EuroLMAI model.”
EuroLMAI is a CEN Workshop Agreement approved by those who participated in its development. The specification build on CEN’s Metadata for Learning Opportunities (MLO) – Advertising (MLO-AD) and its Educational Credit Information Model. It combines information about the learning opportunity taken (MLO-AD), which can be a course or an entire program, and adds information about the learner, the issuer, the results of the learning opportunity, the grading scheme used, language, date, and additional information. The document may indicate achievement of a qualification or credit. The specification may be used as a lightweight transcript. The data is at a fairly course level of granularity, much coarser than the information discussed during the working group’s data analysis.
The Europass Diploma Supplement Application Profile of the EuroLMAI (EuroLMAI Europass DS AP) does add some further details to the data model, including the main field of study, any title conferred, and professional status. There are no detailed reports on competencies obtained.
HR-XML Assessment Report
“A document carrying information about an assessment result. Provides scores and other details to fulfill or respond to an assessment order. AssessmentStatusDetails is part of the data model so that tests, whether completed or pending, can be reported upon.”
The HR-XML assessment result does have a way of indicating the competency assessed, the proficiency level (expressed as a score, point scale, or mark), and additional details determined by the implementer if desired. Scores include a code, interval, minimum, and maximum. Scores may be percentile, raw, stanine, sten, T-score, Z-score, Intelligence Quotient, Scaled, normal curve, or GPA. Scores may be represented as text as well. The specification also permits the indication of a “band,” a place on a normal distribution of scores. Comments may be added, and attachments are supported.
“ A result from an employee performance management (EPM) process or ‘appraisal’. An appraisal typically consists of a number of sections, including an overall rating or score, competency ratings, objective ratings, and other ratings (e.g., ‘core values’ is an example of a section within some appraisals outside of the competency or objectives section). Also includes result metadata, such as appraisal dates, ‘prepared by,’ etc. Note that EPM is an HR-XML approved abbreviation for "Employee Performance Management".”
A note about IP: “Licensee may not facilitate or assist any HR technology Standards Organization other than Licensor in co-opting, copying, distributing, publishing or using the Work without prior written approval of Licensor. Licensee may not create any Derivative Work for ownership by, presentation by, distributing by, publishing by or attribution to any Standards Organization or similar entity other than Licensor, unless specifically approved in writing by Licensor.” If we were interested in leveraging EPM, we would clarify our ability to create derivative works, and provide appropriate attribution to all parties, in writing with HR-XML.
HR-XML Position Competency Model
“A collection of competency information associated with a position or job. A position competency model includes references to individual competencies and references to groups of competencies that are required or associated with a position or job. Within those groups and individual competency references is information about associated proficiency levels (required or desired proficiency), and weightings of the group or individual competencies among their sibilings.”
Includes desired proficiency, required proficiency, level of difficulty, measurable dimensions of the competence (ie frequency), evidence requirements, and weight among sibling requirements, and related competencies. This is fairly different than the MedBiquitous notion of a competency framework in that many more details about the importance of competencies and their relation to a specific job are detailed.
The IMS Learner Information Package
IMS Learner Information Package (IMS LIP) provides a common format for the following types of learner information: accessibilities; activities; affiliations; competencies; goals; identifications; interests; qualifications, certifications and licences; relationship; security keys; and transcripts. IMS LIP is able to indicate the level of a qualification attained, for example, journeyman pipe fitter. The Competencies section is able to indicate skills a learner has acquired and their relationship to other activities or qualifications captured in the LIP. It also includes transcript data. The structure is abstract and fairly general. While IMS LIP does allow one to state attainment of a competence, it does not provide a way of indicating the level of performance at a particular point in time.
IMS also has a Learning Information Services development project which allows for outcomes management within an institution, but the details of that project are only visible to IMS members. (See: http://www.imsglobal.org/lis.html )
MedBiquitous Competency Framework
PESC College Transcript
PESC offers specifications for both high school and college level transcripts. The transcript includes academic awards and honors, degree requirements, academic summary (ie GPA and program details), course grades, additional student achievements, immunizations, and test scores. PESc participants are based in the US and Canada; it’s unclear how well the specification would work for colleges outside of North America. There are codes for describing the area of study for a course. It’s not clear whether grades like “proficient” are able to be communicated using this specification. Notes may be included. Peer benchmark data is not included in the specification. There is an Education Record User Group which focuses on utilization and development of standardized, electronic transcripts.
PESC Education Test Score report
“The Education Test Score Reporting standard can be used by any testing agency to fully communicate test score information to high schools, colleges and universities, affiliated third parties (clearinghouses, employers, research facilities, etc.), and local, state and federal agencies.”
PESC Education test Score Result provides a detailed specification for the communication of test scores. It includes capability to provide subtest scores (ie verbal scores on SATs). It also includes different types of scores: raw score, percent correct, scale score, labels describing a score (ie proficient), mastery scores (ie pass fail), academic grade, gpa, norm referenced, and scores using local scoring systems (ie not otherwise defined). Other student characteristics may be included. While there is no explicit reference to competencies, the subtest code could be used to reference competencies.
PESC Academic ePortfolio Workgroup
PESC also has a workgroup around the development of Academic ePortfolios. It’s not clear if these groups have draft specifications that we could build on. A description of the vision behind the work follows. Dana
“Sophisticated ePortfolio software is being utilized today for integrating course assignments with learning outcomes and academic standards. The assignments are scored using well-designed rubrics, and the data from these scores are then aggregated and disaggregated into reports showing how individuals, courses, programs and departments are performing on a wide variety of student learning outcomes. In addition to helping to manage accreditation systems, these reports help to drive positive change in the way teachers teach and students learn.”
“Taking ePortfolios one step further is the notion of "Folio Thinking," a concept proposed by Helen Chen at Stanford. It is a unique teaching approach that is based on supporting students who are creating their own ePortfolios with opportunities to reflect on the experience through coaching and other techniques. This process both enhances and authentically displays distinct levels of student achievement, knowledge and skills that ultimately will help learner's transition into meaningful work and career development after they graduate.”
Don’t forget assessment methodologies from curriculum inventory
Use of Competencies