Music generation with deep learning techniques

Abstract

This report demonstrated the use of conditioning inputs, together with an appropriate model architecture, to improve the structure of generated music through deep learning. Existing challenges to generate music using deep learning, in particular structure, were reviewed. The use of bar counter, occurrence of repeated motifs, and form of a piece as conditioning inputs were hypothesized to capture long-term structure of music. Then, the proposed model was designed using Bidirectional Long Short-Term Memory (Bi-LSTM) and attention layers to take in the conditioning inputs. To evaluate performance of the proposed model, quantitative analysis was done on the proposed model, the same model without conditioning inputs, and a baseline LSTM model. Following which, a user study was conducted to compare music samples generated by the 3 models. Evaluation results verified that by utilising the 3 conditioning inputs, the proposed model generated more pleasant-sounding and structurally coherent music.