Interactive music visualization

Abstract

Our project aims to revolutionize how violinists learn and perfect their craft through an interactive feedback system that analyzes performances and provides actionable visual feedback so that users can immediately see the mistakes made. The system leverages an Audio Spectrogram Transformer (AST) model, pre-trained on ImageNet and refined using a carefully selected dataset of 1,400 annotated violin audio samples. The system works by converting audio input into spectrograms (visual representations of sound) and analyzing them to detect specific performance elements that need improvements. Early testing shows promising results: our model can identify issues like pitch inaccuracies, improper bowing techniques, and other technical problems as the music is played. This creates a powerful learning tool that benefits both students seeking to improve and instructors looking to provide more precise guidance. This technology bridges traditional music instruction with modern technology. By providing data-driven insights during practice sessions, violinists can develop a deeper understanding of their playing habits and make targeted improvements. Rather than replacing conventional teaching methods, our system enhances them with objective analysis that might otherwise take years of experience to develop.