Web development of job finder – a job scraping and job recommendation platform

Abstract

In today’s fast-paced job market, it can be challenging for job seekers to find suitable opportunities that align with their skills and experiences. While numerous job websites exist, each platform features different job listings, making it time-consuming for users to search across multiple sources. This fragmentation leads to inefficiencies, where job seekers may miss out on relevant opportunities simply because they are posted on a site they do not frequently visit. To address this issue, this project presents Job Finder – A Job Scraping and Job Recommendation Platform that aggregates job listings from multiple job boards, providing users with a centralized and personalized job search experience. The platform utilizes web scraping techniques to extract job postings from various sources, ensuring comprehensive and up-to-date listings. Additionally, a recommendation engine matches jobs to users based on their resumes, considering factors such as skills, experience, and location preferences. By automating job data collection and enhancing job search personalization, the system streamlines the job-hunting process, reducing manual effort and improving job-matching accuracy. Hence, the focus of this Final Year Project (FYP) is to research, design, and implement Job Finder, followed by testing and evaluation. Research efforts involved analyzing existing job listing platforms, studying web scraping techniques, exploring recommendation system methodologies, and evaluating suitable technologies for building a scalable and efficient system. After reviewing various technology options, the selected technology stack included Node.js for backend development, utilizing Express for API management. Python, along with Flask, was chosen for web scraping and handling API requests. The frontend was developed using HTML and JavaScript, with NoSQL serving as the database for storing job listings and user profiles. The research phase also involved investigating text-processing techniques for resume-based job matching, ensuring that job recommendations are tailored to users’ qualifications. Despite the steep learning curve associated with the chosen technology stack during the implementation phase, a fully functional Job Finder Platform was successfully developed, meeting the project’s requirements. The platform effectively aggregates job listings from multiple sources and provides personalized job recommendations based on user profiles and resumes. Following the implementation, several potential enhancements were identified to further improve the platform. These include enabling users to apply for jobs directly through the platform and allowing job seekers to leave reviews on companies based on their experiences with employers, providing valuable insights for future applicants.