At the invitation of the European Training Foundation's department of Skills analysis and anticipation, the ESCO team presented the recent developments on the use of artificial intelligence for the maintenance of ESCO in ETF's online capacity building programme on big data for labour market analysis organised online on the 15th of November 2022.

This is in fact the continuation of a long-standing and fruitful cooperation between ETF and the Commission's services working on ESCO, since the early days of the Make it Match project of the Eastern partnership (2018).

From the start of ETF project Big Data for Labour Market Intelligence (LMI), ESCO has been the taxonomy used for the classification of skills from online job vacancies; and together with ISCO, ESCO is also used for the occupational classification of this data. Moreover, the new version of ESCO (1.1) and the new ESCO portal launched in 2022 are taking a more prominent role in the knowledge-sharing and debates with country and experts’ community of the ETF project.

As a European standard taxonomy of occupations and skills, ESCO is being used by ETF in the development of an innovative data system for the analysis of skills demand coming from online job vacancies. In the first phase (2019-2021) the project achieved tangible results, notably the publication of an introductory Methodological Handbook, establishment of the databases and online visualisation platforms (dashboards) in three countries (Ukraine, Tunisia, Georgia), analytical reports and capacity development programs conducted every year. In 2022 the project started a new phase, marked by the inclusion of additional countries in the data system (Egypt and Kenya), an upgrade of data quality and exploration of new themes and data sources. ETF works with a specialised consortium of data scientists and researchers (LightCast and CRISP – University Milano Bicocca). ETF project gathers experts and revelevant institutions from different countries in Europe (such as North Macedonia, Serbia, Armenia, Azerbaijan, Moldova, Georgia, Ukraine) and from Africa (Egypt, Tunisia, Mauritius, Kenya, Sierra Leone, South Africa, and others).

The ESCO team contributed to the last capacity development program of ETF Project, comprising six webinars, held from 2/Nov to 6/Dec. This presentation provided information on ESCO structure, concepts, use cases and tools and introduced a new theme for this ETF community: AI applications in ESCO.

This is a hands-on learning programme that ETF is running since 2019. The main objective of the capacity building programme 2022 was twofold:

• to provide technical training on concepts, methodological framework, and practical applications of big data in the labour market and,
• to showcase and disseminate the value of landscaping and ranking of online job vacancy sources,of new themes of analysis (green skills, remote work); of the importance of using big data in relation to the wider labour market statistics; of web data sources for supply side analysis and the role of AI in the data system.

During the extended period of the capacity building programme (November-December 2022), participants learned about the complexity and usefulness of using big data to produce labour market data and analyse demand (with a focus on online job vacancies) and how standard taxonomies like ESCO (but also ISCO, NACE and NUTS) and new techniques such as machine learning are fundamental in identifying and visualising relevant insights on skills and occupational features in each specific country. 

Moreover, very interesting research has been developed by CRISP in an attempt to link big data and traditional survey data using AI to study skills mismatch. A concrete use case was building a crosswalk between the Program for the International Assessment of Adult Competences (PIAC) data as a proxy for identifying the skills supply and ESCO skills which are used to analyse the skills demand coming from online vacancies. The linkage is done using AI in a framework that combines various methods such as embeddings, taxonomy alignment and experts’ validation. More information about the crosswalk and how to access it freely can be found on the website. 

For more information, please visit the training webpage to access all presentations of the full training programme, and to get more information on the technical and methodological outputs of the ETF project, such as the Handbook, analytical reports and all materials of the previous training programmes. Moreover, in the YouTube channel Big Data for LMI you find over 40 training videos, including the videos of the 2022 programme.

For more information on ETF Big Data LMI project, please contact: Eduarda Castel-Branco, ecb@etf.europa.eu