Neural architecture search

Abstract

Neural Architecture Search is a technique for designing neural network architectures with minimal human intervention by allowing an algorithm to search through an architecture space to find an optimal architecture design. Without the limitations imposed by a designer’s prior knowledge, NAS techniques have found architectures that outperform the best human designed architectures. However, NAS techniques require high computational costs which can limit their practical utility. Furthermore, there is a lack of a feedback mechanism during the training process for researchers to evaluate the progress. Thus, the goal of this project is to increase the practical utility of NAS. To this end, a technique to actively select an informative subset of the available dataset for training is presented. This is expected to decrease training time as the number of training data samples is reduced. A dashboard for evaluating the performance of neural networks during the training process is also designed to allow researchers to track a model’s progress. Finally, a novel NAS strategy with Neural Processes (NP-NAS) is proposed and its advantages over existing techniques are empirically verified.