Music Segmentation and Boundary Detection

Work by Luke Barrington, Antoni Chan and Prof. Gert Lanckriet

We investigate a Dynamic Texture Mixture (DTM) model of music that considers a musical signal to be a structured sequence of coherent textures or segments. A complete song transitions between a mixture of these textures. The DTM model can both detect transition boundaries and accurately cluster coherent segments. The similarities between the dynamic textures which define these segments are based on both timbral and rhythmic qualities of the music, indicating that the DTM model simultaneously captures two of the important aspects required for automatic music analysis.


Example of the true and DTM model segmentation of song p012m from the RWC dataset.
See more examples


Locally Linear Embedding 2-D visualization of the distribution of song segments.
Each black dot is a song segment. Seven songs are highlighted in different colors, with segments marked as Ο (verse), ☐ (chorus), ◊ (bridge), and Δ (“other”).
Listen to similar segments for each song from the RWC database.


Automatically-generated tags describing each of the automatically-determined segments of the Queen song "Boehmian Rhapsody". The "Bohemian Rhapsody problem" when a song has multiple passages that have very different semantic descriptions can be solved by automatic segmentation. Now each segment has a homogenous, consistent description. Y-axis labels are added to remined the reader of the song structure
Listen to the 35-second version of Bohemian Rhapsody!


Automatically-generated tags describing each of the automatically-determined segments of the Led Zeppelin song "Stairway to Heaven". Y-axis labels are added to remined the reader of the song structure

Relevant Publications
L. Barrington, A.B. Chan, G. Lanckriet - Modeling Music as a Dynamic Texture. IEEE Transactions on Audio, Speech and Language Processing 18-3 pp 602-612.
Python code for the algorithm described in this paper is available from Luke Barrington's website or here.
Barrington, Chan and Lanckriet. Dynamic Texture Models of Music. In ICASSP 2009 slides
Chan and Vasconcelos - Modeling, clustering, and segmenting video with mixtures of dynamic textures. IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 909–926, May 2008.