Difference between revisions of "2009:Audio Chord Detection Results"

From MIREX Wiki
(Team ID)
Line 15: Line 15:
 
====Team ID====
 
====Team ID====
  
'''CH''' =  [[Christopher Harte]]<br />
+
'''CH''' =  [https://music-ir.org/mirex/2009/results/abs/ https://music-ir.org/mirex/2009/results/abs/ Christopher Harte]<br />
'''DE''' = [[Dan Ellis]]<br />
+
'''DE''' = [https://music-ir.org/mirex/2009/results/abs/ Dan Ellis]<br />
'''KO1''' =  [[Maksim Khadkevich, Maurizio Omologo]]<br />
+
'''KO1''' =  [https://music-ir.org/mirex/2009/results/abs/ Maksim Khadkevich, Maurizio Omologo]<br />
'''KO2''' =  [[Maksim Khadkevich, Maurizio Omologo]]<br />
+
'''KO2''' =  [https://music-ir.org/mirex/2009/results/abs/ Maksim Khadkevich, Maurizio Omologo]<br />
'''MD''' =  [[Matthias Mauch, Simon Dixon]]<br />
+
'''MD''' =  [https://music-ir.org/mirex/2009/results/abs/ Matthias Mauch, Simon Dixon]<br />
'''OGF1''' = [[Laurent Oudre,Yves Grenier,Cedric Fevotte]]<br />
+
'''OGF''' = [https://music-ir.org/mirex/2009/results/abs/ Laurent Oudre,Yves Grenier,Cedric Fevotte]<br />
'''OGF2''' = [[Laurent Oudre,Yves Grenier,Cedric Fevotte]]<br />
+
'''PP'''  = [https://music-ir.org/mirex/2009/results/abs/ Helene Papadopoulos, Geoffroy Peeters]<br />
'''PP'''  = [[Helene Papadopoulos, Geoffroy Peeters]]<br />
+
'''PVM1'''  = [https://music-ir.org/mirex/2009/results/abs/ Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens]<br />
'''PVM1'''  = [[Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens]]<br />
+
'''PVM2'''  = [https://music-ir.org/mirex/2009/results/abs/ Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens]<br />
'''PVM2'''  = [[Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens]]<br />
+
'''RRHS1''' = [https://music-ir.org/mirex/2009/results/abs/ Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]<br />
'''RRHS1''' = [[Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]]<br />
+
'''RSHS2''' = [https://music-ir.org/mirex/2009/results/abs/ Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]<br />
'''RSHS2''' = [[Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]]<br />
+
'''RSHS3''' = [https://music-ir.org/mirex/2009/results/abs/ Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]<br />
'''RSHS3''' = [[Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]]<br />
+
'''RUSUSL''' = [https://music-ir.org/mirex/2009/results/abs/ J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee]<br />
'''RUSUSL''' = [[J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee]]<br />
+
'''WEJ1''' = [https://music-ir.org/mirex/2009/results/abs/ Adrian Weller, Daniel Ellis, Tony Jebara]<br />
'''WEJ1''' = [[Adrian Weller, Daniel Ellis, Tony Jebara]]<br />
+
'''WEJ2''' = [https://music-ir.org/mirex/2009/results/abs/ Adrian Weller, Daniel Ellis, Tony Jebara]<br />
'''WEJ2''' = [[Adrian Weller, Daniel Ellis, Tony Jebara]]<br />
+
'''WEJ3''' = [https://music-ir.org/mirex/2009/results/abs/ Adrian Weller, Daniel Ellis, Tony Jebara]<br />
'''WEJ3''' = [[Adrian Weller, Daniel Ellis, Tony Jebara]]<br />
+
'''WEJ4''' = [https://music-ir.org/mirex/2009/results/abs/ Adrian Weller, Daniel Ellis, Tony Jebara]<br />
'''WEJ4''' = [[Adrian Weller, Daniel Ellis, Tony Jebara]]<br />
+
'''WEJ5''' = [https://music-ir.org/mirex/2009/results/abs/ Adrian Weller, Daniel Ellis, Tony Jebara]<br />
'''WEJ5''' = [[Adrian Weller, Daniel Ellis, Tony Jebara]]<br />
+
'''WEJ6''' = [https://music-ir.org/mirex/2009/results/abs/ Adrian Weller, Daniel Ellis, Tony Jebara]<br />
'''WEJ6''' = [[Adrian Weller, Daniel Ellis, Tony Jebara]]<br />
+
'''WEJ7''' = [https://music-ir.org/mirex/2009/results/abs/ Adrian Weller, Daniel Ellis, Tony Jebara]<br />
'''WEJ7''' = [[Adrian Weller, Daniel Ellis, Tony Jebara]]<br />
 
  
 
== Overall Summary Results ==
 
== Overall Summary Results ==

Revision as of 16:23, 16 October 2009

Introduction

These are the results for the 2008 running of the Audio Chord Detection task set. For background information about this task set please refer to the Audio Chord Detection page.

Task Descriptions

Task 1 (Pretrained Systems) Go to Task 1 Results:


Task 2 (Train-Test Systems) Go to Task 2 Results: System trained on ~2/3 of the dataset and tested on ~1/3. Album filtering was applied on each train-test fold such that the songs from the same album can not appear in both train and test sets simultaneously.

Overlap score was calculated as the ratio between the overlap of the ground truth and detected chords and ground truth duration. Also a secondary overlap score was calculated by ignoring the major-minor variations of the detected chord (e.g., C major == C minor, etc.).

General Legend

Team ID

CH = https://music-ir.org/mirex/2009/results/abs/ Christopher Harte
DE = Dan Ellis
KO1 = Maksim Khadkevich, Maurizio Omologo
KO2 = Maksim Khadkevich, Maurizio Omologo
MD = Matthias Mauch, Simon Dixon
OGF = Laurent Oudre,Yves Grenier,Cedric Fevotte
PP = Helene Papadopoulos, Geoffroy Peeters
PVM1 = Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens
PVM2 = Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens
RRHS1 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
RSHS2 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
RSHS3 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
RUSUSL = J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee
WEJ1 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ2 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ3 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ4 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ5 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ6 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ7 = Adrian Weller, Daniel Ellis, Tony Jebara

Overall Summary Results

Task 1 Results

Task 1 Overall Results

file /nema-raid/www/mirex/results/chord/task1/summary/task1.overall.csv not found

Task 1 Run Times

TBA file /nema-raid/www/mirex/results/chord/task1/task1_runtimes.csv not found

Task 1 Friedman's Test for Significant Differences

The Tukey-Kramer HSD multi-comparison data below was generate using the following MATLAB instruction.

Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05); file /nema-raid/www/mirex/results/chord/task1/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask1.friedman.Friedman_TukeyKramerHSD.csv not found

https://music-ir.org/mirex/2009/results/chord/task1/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask1.friedman.Friedman_Mean_Ranks.png

Task 2 Results

Task 2 Overall Results

file /nema-raid/www/mirex/results/chord/task2/summary/task2.overall.csv not found

Task 2 Run Times

TBA file /nema-raid/www/mirex/results/chord/task2/task2_runtimes.csv not found

Task 2 Friedman's Test for Significant Differences

The Tukey-Kramer HSD multi-comparison data below was generate using the following MATLAB instruction.

Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05); file /nema-raid/www/mirex/results/chord/task2/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask2.friedman.Friedman_TukeyKramerHSD.csv not found

https://music-ir.org/mirex/2009/results/chord/task2/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask2.friedman.Friedman_Mean_Ranks.png


Individual Results

Task1

CH = Christopher Harte
DE = Dan Ellis
KO1 = Maksim Khadkevich, Maurizio Omologo
KO2 = Maksim Khadkevich, Maurizio Omologo
MD = Matthias Mauch, Simon Dixon
OGF = Laurent Oudre,Yves Grenier,Cedric Fevotte
PP = Helene Papadopoulos, Geoffroy Peeters
PVM1 = Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens
PVM2 = Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens
RRHS1 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
RSHS2 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
RSHS3 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh

Task2

RUSUSL = J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee
WEJ1 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ2 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ3 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ4 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ5 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ6 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ7 = Adrian Weller, Daniel Ellis, Tony Jebara