This paper discusses the evaluation of skill classifiers applied to job advertisement data, an increasingly important source of information on labour market skill demand. Skill classifiers (algorithms that automatically extract required skills from job ads) are widely used to generate statistics and support applications such as labour market analysis and job matching. This paper focuses on classifiers based on ESCO's taxonomy. Despite their growing use, systematic evaluation of skill classifiers remains limited in the literature and in practice. This is largely due to two key challenges: the scale of the ESCO taxonomy (approximately 14,000 skills) and the strong class imbalance in skill data. To address these challenges, this paper proposes a methodology based on stratified and clustered sampling of skill–job advertisement pairs, combined with expert annotation and bootstrapping for inference. The results show that the main limitation of skill classifiers lies in incomplete detection of relevant skills (false negatives), rather than incorrect predictions. In addition, text-matching approaches are found to perform competitively relative to large language models, and different classifiers capture partially overlapping sets of skills.

Read the full study: "Evaluating ESCO skill classifiers"

This article contributes to the broader collection of external ESCO publications, showcasing the use of ESCO within various methodologies or its presentation in both European and International contexts. As ESCO becomes increasingly used in applications and research projects across Europe and beyond, it is valuable to collect such sources and share best practices by diverse stakeholders. Therefore, this collection of external publications strengthens the exchange of knowledge within the ESCO community and can contribute to mutual learning in the field of skills, occupations and qualifications among European and international actors. If you are interested in sharing your publication, please write to EMPL-ESCO-SECRETARIAT@ec.europa.eu