Measuring professional competences

Short description:

The ASCOT project (Technology-based Assessment of Skills and Competences in Vocational Education and Training; https://www.ascot-vet.net/ascot/de/ueber-ascot/forschungsinitiative-ascot-2011-2015/forschungsinitiative-ascot-2011-2015_node ) funded by the Federal Ministry of Education and Research focused on the visualisation of vocational competences. In the process, competence models and technology-based competence measurement procedures were developed on the basis of international standards for relevant occupations in the fields of technology, health and business.

Three levels of competences were taken into account: (1) the vocational (domain-specific) competences, which were the focus of the analysis, (2) the occupation-specific social and communicative competences including problem-solving competences, and (3) the general competences such as reading, mathematical and scientific competences. In addition, systematic and individual contextual factors were also surveyed in order to obtain indications of the conditions for the successful development of competences in dual VET.

The aim of our joint project CoBALIT (Competencies in the Field of Business and Administration, Learning, Instruction, and Transition), which we carried out together with Prof. Esther Winter and Prof. Clemens Draxler and their teams, was to develop a technology-based performance assessment that makes it possible to describe and measure an intrapreneurship competence of industrial clerks at the end of their three-year vocational training, which is acquired within the framework of dual training. This training occupation is considered a so-called "cross-sectional occupation" and can be understood as exemplary for the area of commercial vocational training.

Here we followed the evidence-centred design approach with four central design components:

• Domain analysis and domain modelling

• Item design and implementation in a test environment

• Definition of an accepted solution space and creation of a scoring guide

• Empirical verification of the theoretically formulated competency model (incl. IRT scaling)

• Germany-wide main survey in testlet design

Selected results:

• Industrial clerks at the end of their three-year vocational training already have good to very good competences in the area of planning and implementing innovative ideas. However, the creative aspect of idea development (incl. seizing intrapreneurship opportunities) could be improved.

• In addition to previous education, age and family educational background, the conditions for success in promoting intrapreneurship in the context of dual training for industrial clerks are, above all, the climate of innovation and the learning opportunities received in the company.

Literature:

• Weber, S., Draxler, C., Achtenhagen, F., Bley, S., Wiethe-Körprich, M., Kreuzer, C., & Gürer, C. (2021). Measuring Apprentices‘ Intrapreneurship Competence in Vocational Education and Training (VET): An Interdisziplinary Model-Based Assessment. In E. Kuusisto, M. Ubani, P. Nokelainen & A. Toom (Eds.), Good Teachers for Tomorrow’s Schools. (pp. 257-279). Brill/Sense.

• Kreuzer, C., & Weber, S. (2020). Apprentices’ Errors in Intrapreneurship: Findings from a Large-Scale Assessment. Vocations and Learning, 13, 341-366.

• Weber S., Draxler, C., Bley, S., Wiethe-Körprich, M., Weiß, C., & Gürer, C. (2016). Der Projektverbund CoBALIT: Large Scale-Assessments in der kaufmännischen Berufsbildung – Intrapreneurship (CoBALIT). In K. Beck, M. Landenberger, & F. Oser (Eds.), Technologiebasierte Kompetenzmessung in der beruflichen Bildung – Resultate aus dem Forschungsprogramm ASCOT (pp. 75–92). Gütersloh: Bertelsmann.

• Weber, S., Wiethe-Körprich, M, Bley. S., Weiß, C., Draxler, C., & Gürer, C. (2016). Modellierung und Validierung eines Intrapreneurship-Kompetenz-Modells bei Industriekaufleuten. Unterrichtswissenschaft, 44(2), 149–168.

• Weber, S., Wiethe-Körprich, M., Bley, S., Weiß C., & Achtenhagen, F. (2015). Intrapreneurship-Verhalten an kaufmännischen Arbeitsplätzen – Analysen von Projektberichten. Empirische Pädagogik 28(1), 84-105.

• Trost S., & Weber, S. (2012). Fähigkeitsanforderungen an kaufmännische Fachkräfte. Eine kompetenzbasierte Analyse von Stellenanzeigen mittels O*NET. Zeitschrift für Berufs- und Wirtschaftspädagogik, 108(2), 217-242.

Enlarge

Technologiebasierte Kompetenzmessung in der beruflichen Bildung (ASCOT)

https://bmbf-prod.bmbfcluster.de/upload_filestore/pub/ASCOT.pdf

Enlarge

Beck, K., Landenberger, M., & Oser, F. (2016) (Hrsg.), Technologiebasierte Kompetenzmessung in der beruflichen Bildung – Resultate aus dem Forschungsprogramm ASCOT. Bielefeld: Bertelsmann.

Other projects to measure professional competences:

• Kreuzer, C., Birk, C., Weber, S., & Baggen, Y. (2019). Visualizing Opportunity Recognition Competence. Conference Paper at the EARLI (European Association for Research on Learning and Instruction), Aachen, Germany.

• Weber, S., & Achtenhagen, F. (2019). Measuring Innovative Thinking and Acting Skills as Workplace-Related Professional Competence. In V. Hammler Kenon & S. Vasant Palsole (Eds.), The Wiley Handbook of Global Workplace Learning (pp. 121-139). New York et al.: Wiley.

• George, A. C., Bley, S. & Pellegrino, J. (2019). Characterizing and Diagnosing Complex Professional Competencies – An Example of Intrapreneurship. Educational Measurement: Issues and Practice. 38(2), 89-100. DOI: 10.1111/emip.12257

• Sangmeister, J., Winther, E., Deutscher, V., Bley, S., Kreuzer, C. & Weber, S. (2018). Designing Competence Assessment in VET for a Digital Future. In D. Ifenthaler (Ed.), Digital Workplace Learning (pp. 65-92). Cham, CH: Springer.

• Kreuzer, C., & Weber, S. (2018). Modelling Opportunity Recognition competence as a foundation for teaching and learning in vocational education. Vocations and Learning, 11(1)1-25.

• Kreuzer, C. (2018). Visualisierung der Opportunity Recognition-Kompetenz von Industriekaufleuten. Zeitschrift für Berufs- und Wirtschaftspädagogik, 114(2), 247-27.1

• Bley, S. & George A. C. (2017). Kognitive Diagnosemodelle zur Begleitung individualisierter Lehr- und Lernprozesse: Neue Möglichkeiten aus einem alternativen Ansatz? Zeitschrift für Berufs- und Wirtschaftspädagogik, 113(1), 56-85.

• Bley, S. (2017). Developing and validating a technology-based diagnostic assessment using the evidence-centered game design approach: An example of intrapreneurship competence. Empirical Research in Vocational Education and Training, 9(6), 1-32. https://doi.org/10.1186/s40461-017-0049-0

• Wiethe-Körprich, M., Weber, S., Bley, S., & Kreuzer, C. (2017). Intrapreneurship competence as a manifestation of work agency: A systematic literature review. In M. Goller & S. Paloniemi (Eds.), Agency at work: An agentic perspective on professional learning and development (pp. 37-65). Cham, CH: Springer.

• Kreuzer, C., Weber, S., Bley, S., & Wiethe-Körprich, M. (2017). Measuring Intrapreneurship Competence as Manifestation of Work Agency in Different Educational Settings. In M. Goller & S. Paloniemi (Eds.), Agency at work. An agentic perspective on professional learning and development (pp. 373-399). Cham, CH: Springer.

• Guggemos, J., & Schönlein, M. (2017). Arbeitsaufgaben von Industriekaufleuten im Externen Rechnungswesen: Analyse und Modellierung der Domäne. Zeitschrift für Berufs- und Wirtschaftspädagogik, 113(2), 325-347.

• Guggemos, J. & Schönlein, M. (2015). Modellierung von Kompetenzen in der beruflichen Bildung - Entwicklung und Validierung eines Kompetenzniveaumodells für das externe Rechnungswesen. Zeitschrift für Berufs- und Wirtschaftspädagogik, 111(4), 524-551.

• Bley, S., Wiethe-Körprich, M. & Weber, S. (2015). Formen kognitiver Belastung bei der Bewältigung technologiebasierter authentischer Testaufgaben – eine Validierungsstudie zur Abbildung von beruflicher-Kompetenz. Zeitschrift für Berufs- und Wirtschaftspädagogik, 111(2), 268-294

Contact:

Prof. Dr. Susanne Weber
Institute for Human Resource Education & Management
Munich School of Management
Ludwig-Maximilians University Munich