Analyzing job roles and skill description using NLP techniques (part 2: applying machine learning techniques) – collaboration with ARISE and SSG

Abstract

This project presents a novel approach to tackling the lack of consistency within the job market in order to enhance career transition strategies within the workplace. There are 2 primary objectives within this project, the first objective involves the classification of online job descriptions into a coherent framework, delineating the requisite skills for each role. Utilizing datasets provided by ARISE SG, such as 'Taxonomy.xlsx' and 'sampled1000.xlsx', this study implements natural language processing (NLP) techniques to create a correlation between job titles and their associated skill sets, despite variances across different corporate interpretations. The second objective seeks to quantitatively assess the difficulty of transitioning between job roles using the Skills Framework Database. Various methodologies were employed within this project such as data preprocessing to eliminate inconsistencies, application of a BERT transformer model for data labeling, and development of a machine learning model for skill association and predictive analytics. Through this project, we aim to not only provide more insights into the current job market but also develop a strategic tool for career development in the rapidly evolving job market.