Content-based sports video analysis and composition

Abstract

This thesis proposes solutions for content-based sports video analysis, including multi-modal feature extraction, middle-level representation and semantic event detection. In addition, solutions for sports video composition and personalization are also examined. The first part of the thesis describes our methodology to detect semantic events and event boundaries from both broadcast sports video and non-broadcast sports video. Specifically, to process broadcast sports video, we analyze both the visual/audio features and the associated web-casting text information to detect event, locate event boundaries and identify involved players/teams; To process non-broadcast sports video, we select the raw unedited main-camera soccer video as the input and use visual, audio and motion features extraction with multi-level modeling to detect event and event boundaries. The second part of the thesis introduces three novel applications based on our proposed sports video analysis techniques. The first application is a live sports highlight generation system; The second application attempts to automatically generate broadcast soccer video composition from multiple raw captures; The third application is a personalized music sports video generation system to automatically select and align desired sports video scenes with music clips. The three proposed systems are tested using objective evaluations and subjective user studies.