Please use this identifier to cite or link to this item:
Title: Building generalizable deep learning solutions for mobile sensing
Authors: Xu, Huatao
Keywords: Computer and Information Science
Issue Date: 2024
Publisher: Nanyang Technological University
Source: Xu, H. (2024). Building generalizable deep learning solutions for mobile sensing. Doctoral thesis, Nanyang Technological University, Singapore.
Abstract: In an era where embedded and mobile devices are becoming ubiquitous, the intersection of Artificial Intelligence and the Internet of Things (AIoT) is rapidly transforming various fields. This thesis delves into the challenges and innovations in this domain, particularly focusing on the development of generalized sensing models for wearable sensor data. We introduce novel approaches to leverage the abundant unlabeled sensor data, physical sensing knowledge, and common knowledge embedded in Large Language Models (LLMs) to significantly enhance AIoT models. Firstly, we propose a foundational model for IoT sensor data processing, specifically focusing on Inertial Measurement Unit (IMU) data. This model, named LIMU-BERT, is inspired by self-supervised techniques in natural language processing and adapted for the multi-modal nature of sensor data. LIMU-BERT effectively learns generalizable representations from unlabeled sensor data, demonstrating superior performance in two typical sensing applications. Secondly, we introduce UniHAR, a universal learning framework for mobile devices. This framework employs physics-informed data augmentation techniques to address data heterogeneity in IMU-based human activity recognition. UniHAR, implemented on mobile platforms, showcases remarkable adaptability across various user groups and environments, outperforming existing solutions in cross-dataset model transfers. Lastly, the thesis explores the application of LLMs in AIoT, particularly in mobile sensing. We investigate the potential of LLMs, such as ChatGPT, to process IoT sensor data and accomplish real-world tasks. Our approach, termed "Penetrative AI", extends LLMs' capabilities beyond natural language processing and enables LLMs to interact with the physical world through IoT sensors/actuators, utilizing their embedded common-sense knowledge. The promising results from applications in user activity sensing and human heartbeat detection highlight the potential of LLMs to interpret sensor data and perform physical world tasks. The three works contribute significantly to the field of AIoT by developing novel methodologies that enhance the utility and adaptability of AIoT systems in diverse real-world scenarios.
DOI: 10.32657/10356/174809
Schools: School of Computer Science and Engineering 
Rights: This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Theses

Files in This Item:
File Description SizeFormat 
NTU_thesis_XU HUATAO.pdf16.87 MBAdobe PDFThumbnail

Page view(s)

Updated on Jul 20, 2024


Updated on Jul 20, 2024

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.