Low-resource symbolic music generation using transformers and chord identification heuristic

Abstract

Music generation using computers is a task that while interesting, has received comparatively little attention compared to more mainstream problems. With recent advances in deep learning methods, it affords music generation researchers a new range of tools to apply to the cause. One of these is the Transformer, an Attention-driven model that has had considerable success in modelling sequential data, beating out once state-of-the-art models such as Long-Short Term Memory models. Even though there are existing efforts to apply Transformer models to the music generation task, the application of music domain knowledge is still lacking. Models are still largely trained on data that conveys only pitch, with information underlying the data such as chord progressions largely being unattended to. The purpose of this research is to test if the inclusion of chord progression labels can help improve model performance in the task of music generation, with the goal of producing more realistic and pleasant-sounding pieces. A template-mask method was used to quickly label large sets of pianoroll data, before the data was converted into a MIDI-like format with the inclusion of Chord events to represent chords in a chord progression. A generator Transformer based off the GPT-2 model was then trained on this data. By observing the pieces generated by the model, we were able to identify convincing signs that the model was able to learn the concept of chords and chord progressions, as well as produce pieces that have more recurring structure due to its understanding of these latent structures of music.