Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/171932
Full metadata record
DC FieldValueLanguage
dc.contributor.authorTan, Isaac Jun Hongen_US
dc.date.accessioned2023-11-16T08:53:09Z-
dc.date.available2023-11-16T08:53:09Z-
dc.date.issued2023-
dc.identifier.citationTan, I. J. H. (2023). Collecting and annotating videos that teach MS PowerPoint. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/171932en_US
dc.identifier.urihttps://hdl.handle.net/10356/171932-
dc.description.abstractThe central aim of this project is to generate a comprehensive dataset for training an artificial intelligence (AI) that is able to operate Microsoft PowerPoint autonomously. This project encompasses several different phases: Starting with the identification of videos that teach Microsoft PowerPoint following which we will download the identified videos using Jupyter Notebook with the help of the Pytube library. This is followed by the transcribing of videos that lack closed captions with the Whisper Model. Following this, the annotation process is then executed whereby the keystroke and the mouse clicks are then labeled using Sequence labeling in Doccano. The project then transits into the model training phase where both T5 and FLAN-T5 neural network models are experimented on for their ability to interpret and translate narrated instructions into corresponding mouse and keyboard actions to decide which model would achieve the better performance. Given the limitations of YouTube’s dataset, data augmentation techniques were employed using ChatGPT to improve model training.en_US
dc.language.isoenen_US
dc.publisherNanyang Technological Universityen_US
dc.subjectEngineering::Computer science and engineering::Computing methodologies::Artificial intelligenceen_US
dc.titleCollecting and annotating videos that teach MS PowerPointen_US
dc.typeFinal Year Project (FYP)en_US
dc.contributor.supervisorLi Boyangen_US
dc.contributor.schoolSchool of Computer Science and Engineeringen_US
dc.description.degreeBachelor of Engineering (Computer Science)en_US
dc.contributor.supervisoremailboyang.li@ntu.edu.sgen_US
item.grantfulltextrestricted-
item.fulltextWith Fulltext-
Appears in Collections:SCSE Student Reports (FYP/IA/PA/PI)
Files in This Item:
File Description SizeFormat 
Isaac_Tan_Jun_Hong_Final_Year_Project_Report.pdf
  Restricted Access
Undergraduate project report1.22 MBAdobe PDFView/Open

Page view(s)

108
Updated on Jul 22, 2024

Download(s)

10
Updated on Jul 22, 2024

Google ScholarTM

Check

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.