CALOHEE Phase 2
Rational of the project
The Measuring and Comparing Achievements of Learning Outcomes in Higher Education In Europe (CALOHEE) II project is a follow-up of the CALOHEE project (2016-2018). That project resulted in highly innovative outcomes; that is European Qualifications Reference Frameworks and Assessment Reference Frameworks for five key subject areas, representing as many HE sectors. These Frameworks offer a serious contribution to rethink the internal and external quality assurance policies at European level. CALOHEE offers detailed reference to allow for defining what a relevant and high quality degree programme should entail. Based on a merger of the EQF for LLL and the QF for the EHEA it gives clear indicators for three succeeding levels of learning for both the first and the second cycle. It is therefore unique in its kind and highly innovative. The frameworks have been endorsed by all main stakeholders.
Having the Qualifications Reference Frameworks and related Assessment Reference Frameworks in place, which offer both the core what is expected to be learned in terms of knowledge, skills and ‘wider competences’ (autonomy and responsibility), but also allow for profiling and variations, the next step can be made. This step is twofold.
1) To match the frameworks against existing degree programmes and quality assurance systems and procedures. Matching the programmes is a responsibility of the academics involved in the project, matching the systems and procedures that of the (inter)national QA organisations, which are (also) involved in the Advisory Board of CALOHEE II. The outcomes of this process will be precise information about the (in)consistencies between the present situation, which in turn allows for accommodation of practices. This should be perceived as a reality check to allow for a European wide policy intervention both at the level of HE institutions and QA organisations. The operation will – as indicated as one of the objectives if the FLCPs – offer in-depth knowledge on the present and intended future learning outcomes of degree programmes, but also offer insight in the present learning, training and assessment situations. It will highlight present omissions in degree programmes Europe wide. This is of importance for individual HE institutions, but also very relevant for policy makers in education and training at all levels, including the national and European ones.
2) To develop on the basis of the detailed reference frameworks defined, an applicable and fair instrument to measure performance at the end of first cycle studies. This requires the development (and initial testing) of a blueprint/ test items bank which allows for bridging different cultural, disciplinary and educational specificities of national and local entities. It should also allow for differences in profiles and mission and variation in level to be achieved.
Enhancing the evidence of learning
At present, real evidence which allows for comparison of learning/ performance at international level based on an agreed measurement instrument does not exist. The OECD did give it a try with the development of the Assessment of Higher Education Learning Outcomes (AHELO) project, but proved not successful although the feasibility study offered insight in the challenges to be met. Most important lessons learned were: do not pilot at global level, but choose a region which has an appropriate level of consistency regarding educational system/structure such as the E(H)EA; involve the grassroot level from the very start; accept that generic competences/skills and subject specific ones are developed in connotation; develop wide support of all main stakeholders for the model to be defined. And finally apply a step-by-step approach: Find common ground in well-defined subject area specific qualifications reference framework which contains clear reference points widely accepted by the academic field and its users. From this framework develop a much more detailed subject area assessment reference framework. The last to be underpinned with up-to-date LTA strategies and approaches which allow for meeting the learning outcomes (intended level of competences) based on the notions of outcome based and active learning. Find agreement on what should be measured.
These steps have been successfully made in CALOHEE. CALOHEE has also defined already the measurement instrument by making a distinction between four parameters: (1) Knowledge: theory and methodology; (2) Application of knowledge and skills; (3) Preparation for employment and (4) Preparation for civic, social and cultural engagement. This allows for developing an ambitious and sophisticated instrument to measure performance reflecting different categories and levels of learning.
The proposed project gives real subsistence to both general and specific objectives of the FLCPs Call. It promotes innovation in the HE field at the highest imaginable level. Recently developed instruments – the subject area and assessment reference frameworks which are already the most sophisticated in its kind – will be supported by an instrument that will allow for comparative testing and therefore measuring. Even before actual testing takes place its many indicators will allow for identifying strength and weaknesses of every degree programme. It will empower key stakeholders to develop and mainstream policy innovations to boost the quality and relevance of HE. It will also result in better evidenced QA mechanisms. At present QA is based on procedures and peer reviewing, which have serious weaknesses. Practice has shown rather biased/ personalised judgements. More articulated and sophisticated frameworks offer far better refence material to make fair evaluations, both internally and externally.
Regarding specific objectives the main outcomes of this project will be:
- Matching of representative degree programmes against the CALOHEE frameworks, resulting in an inventory of strength and weaknesses, which will offer evidence to fill gaps in learning and teaching to meet the needs of society in local, national and European settings;
- Matching of present quality assurance procedures and the CALOHEE frameworks, which offer way forward to strengthen the reliability of internal and external quality instruments at local, national and international level.
Building on the CALOHEE frameworks, a sophisticated and innovative bank of selected test items will offer insight and detail of the learning graduates are expected to have obtained to operate successfully in society, both in the workplace and as active citizen.