University of Illinois · Graduate School of Library and Information Science · ISRL

2005 MIREX Contest Results - Symbolic Genre Classification (Contest wiki)


Goal: To classify MIDI recordings into genre categories.

Dataset: Two sets of genre categories were used, one consisting of 38 categories and one consisting of  9 categories. Each category was represented by 25 MIDI files.Thus, the 38 genre test contained 950 MIDI files and the 9 genre test contained 225 MIDI files.Test runs were 3-fold crossvalidated with each algorithm tested using identical training and testing data splits.

OVERALL
Rank Participant Mean Hierarchical Classification  Accuracy  Mean Raw Classification Accuracy 
1 McKay & Fujinaga 77.17% 65.28%
2 Basili, Serafini, & Stellato (NB) 72.08%
58.53%
3
Li, M. 67.57%
55.90%
4
Basili, Serafini, & Stellato (J48) 67.14%
53.14%
5 Ponce de Leon & Inesta 37.76%
26.52%


38 Classes
Rank Participant Hierarchical Classification Accuracy Hierarchical Classification Accuracy Std Raw Classification Accuracy Raw Classification Accuracy Std Runtime (s) Machine Confusion Matrix Files
1
McKay & Fujinaga
64.33% 
1.04
46.11%
1.51
3 days
R
MF_38eval.txt
2
Basili, Serafini, & Stellato (NB) 62.60%
0.26
45.05%
0.55
N/A
N/A
BST_NB_38eval.txt
3
Basili, Serafini, & Stellato (J48)
57.61%
1.14
40.95%
1.35
N/A
N/A
BST_J48_38eval.txt
4
Li, M.
54.91%
0.66
39.79%
0.87
15,948
G
L_38eval.txt
5
Ponce de Leon & Inesta
24.84%
1.40
15.26%
1.13
821
L
PI_38eval.txt
 

9 Classes
Rank Participant Hierarchical Classification Accuracy Hierarchical Classification Accuracy Std Raw Classification Accuracy Raw Classification Accuracy Std Runtime (s) Machine Confusion Matrix Files
1
McKay & Fujinaga
90.00% 
0.60
84.44%
1.41
18,375
R
MF_9eval.txt
2
Basili, Serafini, & Stellato (NB)
81.56%
0.76
72.00%
0.88
N/A
N/A
BST_NB_9eval.txt
3
Li, M.  80.22%
1.47
72.00%
2.31
3,777
G
L_9eval.txt
4
Basili, Serafini, & Stellato (J48)
76.67%
1.11
65.33%
1.65
N/A
N/A
BST_J48_9eval.txt
5
Ponce de Leon & Inesta 
50.67%
1.26
37.78%
2.30
197
L
PI_9eval.txt



Maintained by :J Stephen Downie
Comments to : jdownie at uiuc dot edu
Last modified: 28 September 2005