Please use this identifier to cite or link to this item:
Title: Generalizing transfer Bayesian optimization to source-target heterogeneity
Authors: Min, Alan Tan Wei
Gupta, Abhishek
Ong, Yew-Soon
Keywords: Engineering::Computer science and engineering
Issue Date: 2020
Source: Min, A. T. W., Gupta, A. & Ong, Y. (2020). Generalizing transfer Bayesian optimization to source-target heterogeneity. IEEE Transactions On Automation Science and Engineering, 18(4), 1754-1765.
Project: A19C1a0018
Journal: IEEE Transactions on Automation Science and Engineering
Abstract: Black-box optimization algorithms typically start a search from scratch, assuming little prior knowledge about the task at hand. In practice, this approach can be prohibitive for computationally expensive problems, as a large number of costly function evaluations are often needed before a suitable (near-optimal) solution is found. Under this observation, recent efforts have incorporated transfer learning capabilities into sequential model-based Bayesian optimization (BO) solvers, resulting in substantial performance speed-ups by leveraging information from related past problems. However, a common simplifying assumption in existing approaches is that the search spaces of a previously encountered source and the ongoing target task bear the same features and dimensionality, with the difference lying in their respective objective functions. In this article, we present a generalized transfer BO algorithm that relaxes the aforementioned assumption. Our method jointly transforms source features while training probabilistic transfer regression models for the target, thus applying to practical use-cases where (in addition to the difference in objective functions) the number of features could change across the source and target tasks; for example, features can be added and/or removed. The theoretical basis of our proposal is analyzed, and its empirical performance is demonstrated on synthetic benchmark functions as well as in realistic examples spanning engineering design and the automated configuration of a machine learning model. Note to Practitioners-Problems of industrial interest have a tendency of being repetitive in nature. For this reason, domain experts are always in high demand, as they are able to harness their experience of similar problems to come up with fast solutions in difficult situations. However, domain experts are not easy to find. Given this fact, the present paper puts forth a method for automating the process of knowledge extraction (through experiential learning) and transfer across problems in the domain of computationally expensive black-box optimization. The key novelty and motivation of this work lies in enabling the adaptive transfer of knowledge even when the number of features changes across the source and target problems. Our proposed approach is verified experimentally on a range of benchmarks as well as real-world problems of a computationally expensive nature, highlighting the utility of an optimization engine that is able to learn from experience without the need for constant human intervention.
ISSN: 1545-5955
DOI: 10.1109/TASE.2020.3017644
Schools: School of Computer Science and Engineering 
Organisations: Singapore Institute of Manufacturing Technology, A*STAR
Research Centres: Data Science and Artificial Intelligence Research Centre 
Rights: © 2020 IEEE. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

Citations 20

Updated on Jun 3, 2023

Web of ScienceTM
Citations 20

Updated on Jun 6, 2023

Page view(s)

Updated on Jun 8, 2023

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.