Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/54684
Title: Kernel based online learning
Authors: Zhao, Peilin
Keywords: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
Issue Date: 2013
Source: Zhao, P. (2013). Kernel based online learning. Doctoral thesis, Nanyang Technological University, Singapore.
Abstract: Kernel Based Online Learning (KBOL) is an important branch of online learning in machine learning, in which the objective is to optimize the online predictive performance, typically measured by classification accuracy. It enjoys many advantages when solving real-world large-scale applications, such as classification, regression, ranking, and clustering, etc. Although being extensively studied, KBOL can be more practical if several critical problems can be solved. Firstly, most existing KBOL algorithms keep the weights of existing Support Vectors (SVs) unchanged during the whole online learning process, which is too conservative to significantly improve the online accuracy. To overcome this limitation, based on the primal-dual framework, we extend some existing single updating online learning algorithms to "Double Updating Online Learning" (DUOL), which not only updates the weight of the newly added SV, but also simultaneously updates the weight of one existing SV, referred to "auxiliary example". In this thesis, we investigate effective DUOL algorithms for both binary and multi-class online classification tasks. Secondly, most existing KBOL algorithms are designed with unbounded number of support vectors, making them unsuitable for handling large-scale datasets. To overcome this challenge, we investigate the problem of budget KBOL by designing various budget maintenance strategies for the binary DUOL algorithm that aims to constrain the number of support vectors by a predefined budget when learning the kernel-based prediction function in the online learning process. Thirdly, traditional KBOL algorithms mainly aim to maximize the online accuracy, which is not suitable for some real world tasks, where the datasets may be highly imbalanced so that the cost-sensitive metrics are more appropriate. To solve this issue, we propose a family of cost-sensitive online classification algorithms by exploiting the DUOL techniques, which are designed to directly optimize two well-known cost-sensitive measures: (i) maximization of weighted sum of sensitivity and specificity, and (ii) minimization of weighted misclassification cost. Finally, most of traditional KBOL algorithms only learn from one single target domain, which ignore any existing knowledge learnt from other related source domain. This obviously is a waste of resource. To tackle this problem, we investigate an online learning framework called "Online Transfer Learning" (OTL) that aims to transfer knowledge from some source domain to an online learning task on a target domain.
URI: https://hdl.handle.net/10356/54684
DOI: 10.32657/10356/54684
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Theses

Files in This Item:
File Description SizeFormat 
Thesis-PeilinZhao.pdfMain article1.13 MBAdobe PDFThumbnail
View/Open

Page view(s) 20

469
Updated on Jul 27, 2021

Download(s) 5

459
Updated on Jul 27, 2021

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.