2005:Symbolic Genre Classification Results

From MIREX Wiki

Introduction

Goal

To classify MIDI recordings into genre categories.

Dataset

Two sets of genre categories were used, one consisting of 38 categories and one consisting of 9 categories. Each category was represented by 25 MIDI files.Thus, the 38 genre test contained 950 MIDI files and the 9 genre test contained 225 MIDI files.Test runs were 3-fold cross validated with each algorithm tested using identical training and testing data splits.

Results

Overall

OVERALL
Rank Participant Mean Hierarchical Classification Accuracy Mean Raw Classification Accuracy
1 McKay & Fujinaga 77.17% 65.28%
2 Basili, Serafini, & Stellato (NB) 72.08% 58.53%
3 Li, M. 67.57% 55.90%
4 Basili, Serafini, & Stellato (J48) 67.14% 53.14%
5 Ponce de Leon & Inesta 37.76% 26.52%

38 Classes

38 Classes
Rank Participant Hierarchical Classification Accuracy Hierarchical Classification Accuracy Std Raw Classification Accuracy Raw Classification Accuracy Std Runtime (s) Machine Confusion Matrix Files
1 McKay & Fujinaga 64.33% 1.04 46.11% 1.51 3 days R MF_38eval.txt
2 Basili, Serafini, & Stellato (NB) 62.60% 0.26 45.05% 0.55 N/A N/A BST_NB_38eval.txt
3 Basili, Serafini, & Stellato (J48) 57.61% 1.14 40.95% 1.35 N/A N/A BST_J48_38eval.txt
4 Li, M. 54.91% 0.66 39.79% 0.87 15,948 G L_38eval.txt
5 Ponce de Leon & Inesta 24.84% 1.40 15.26% 1.13 821 L PI_38eval.txt

9 Classes

9 Classes
Rank Participant Hierarchical Classification Accuracy Hierarchical Classification Accuracy Std Raw Classification Accuracy Raw Classification Accuracy Std Runtime (s) Machine Confusion Matrix Files
1 McKay & Fujinaga 90.00% 0.60 84.44% 1.41 18,375 R MF_9eval.txt
2 Basili, Serafini, & Stellato (NB) 81.56% 0.76 72.00% 0.88 N/A N/A BST_NB_9eval.txt
3 Li, M. 80.22% 1.47 72.00% 2.31 3,777 G L_9eval.txt
4 Basili, Serafini, & Stellato (J48) 76.67% 1.11 65.33% 1.65 N/A N/A BST_J48_9eval.txt
5 Ponce de Leon & Inesta 50.67% 1.26 37.78% 2.30 197 L PI_9eval.txt