Difference between revisions of "2005:Symbolic Genre Classification Results"

From MIREX Wiki
(Created page with '{| border="1" |- style="background: yellow; text-align: center;" ! colspan="4" | OVERALL |-style="background: yellow;" !Rank !Participant !Mean Hierarchical Classification Accu…')
 
(→‎9 Classes)
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
{| border="1"
+
==Introduction==
 +
===Goal===
 +
To classify MIDI recordings into genre categories.
 +
 
 +
===Dataset===
 +
Two sets of genre categories were used, one consisting of 38 categories and one consisting of  9 categories. Each category was represented by 25 MIDI files.Thus, the 38 genre test contained 950 MIDI files and the 9 genre test contained 225 MIDI files.Test runs were 3-fold cross validated with each algorithm tested using identical training and testing data splits.
 +
 
 +
==Results==
 +
 
 +
===Overall===
 +
{| border="1" cellspacing="0"
 
|- style="background: yellow; text-align: center;"
 
|- style="background: yellow; text-align: center;"
 
! colspan="4" | OVERALL  
 
! colspan="4" | OVERALL  
Line 9: Line 19:
 
|----
 
|----
 
|1
 
|1
−
|McKay & Fujinaga
+
|[https://www.music-ir.org/mirex/abstracts/2005/mckay.pdf McKay & Fujinaga]
−
|77.17%
+
|align="right " | 77.17%
−
|65.28%
+
|align="right " | 65.28%
 
|----
 
|----
 
|2
 
|2
−
|Basili, Serafini, & Stellato (NB)
+
|[https://www.music-ir.org/mirex/abstracts/2005/basili.pdf Basili, Serafini, & Stellato (NB)]
−
|72.08%
+
|align="right " | 72.08%
−
|58.53%
+
|align="right " | 58.53%
 
|----
 
|----
 
|3
 
|3
 
|Li, M.
 
|Li, M.
−
|67.57%
+
|align="right " | 67.57%
−
|55.90%
+
|align="right " | 55.90%
 
|----
 
|----
 
|4
 
|4
−
|Basili, Serafini, & Stellato (J48)
+
|[https://www.music-ir.org/mirex/abstracts/2005/basili.pdf Basili, Serafini, & Stellato (J48)]
−
|67.14%
+
|align="right " | 67.14%
−
|53.14%
+
|align="right " | 53.14%
 
|----
 
|----
 
|5
 
|5
−
|Ponce de Leon & Inesta
+
|[https://www.music-ir.org/mirex/abstracts/2005/ponce.pdf Ponce de Leon & Inesta]
−
|37.76%
+
|align="right " | 37.76%
−
|26.52%
+
|align="right " | 26.52%
 
|----
 
|----
 
|}
 
|}
−
<br>
+
 
−
{| border="1"
+
===38 Classes===
 +
{| border="1" cellspacing="0"
 
|- style="background: yellow; text-align: center;"
 
|- style="background: yellow; text-align: center;"
 
! colspan="9" | 38 Classes  
 
! colspan="9" | 38 Classes  
Line 48: Line 59:
 
!Machine
 
!Machine
 
!Confusion Matrix Files
 
!Confusion Matrix Files
−
|
 
−
|
 
 
|----
 
|----
 
|1
 
|1
 
|McKay &amp; Fujinaga
 
|McKay &amp; Fujinaga
−
|64.33%  
+
|align="right " | 64.33%  
−
|1.04
+
|align="right " | 1.04
−
|46.11%
+
|align="right " | 46.11%
−
|1.51
+
|align="right " | 1.51
−
|3 days
+
|align="right " | 3 days
−
|R
+
|align="right " | R
−
|MF_38eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/MF_38eval.txt MF_38eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|2
 
|2
 
|Basili, Serafini, &amp; Stellato (NB)
 
|Basili, Serafini, &amp; Stellato (NB)
−
|62.60%
+
|align="right " | 62.60%
−
|0.26
+
|align="right " | 0.26
−
|45.05%
+
|align="right " | 45.05%
−
|0.55
+
|align="right " | 0.55
−
|N/A
+
|align="right " | N/A
−
|N/A
+
|align="right " | N/A
−
|BST_NB_38eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/BST_NB_38eval.txt BST_NB_38eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|3
 
|3
 
|Basili, Serafini, &amp; Stellato (J48)
 
|Basili, Serafini, &amp; Stellato (J48)
−
|57.61%
+
|align="right " | 57.61%
−
|1.14
+
|align="right " | 1.14
−
|40.95%
+
|align="right " | 40.95%
−
|1.35
+
|align="right " | 1.35
−
|N/A
+
|align="right " | N/A
−
|N/A
+
|align="right " | N/A
−
|BST_J48_38eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/BST_J48_38eval.txt BST_J48_38eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|4
 
|4
 
|Li, M.
 
|Li, M.
−
|54.91%
+
|align="right " | 54.91%
−
|0.66
+
|align="right " | 0.66
−
|39.79%
+
|align="right " | 39.79%
−
|0.87
+
|align="right " | 0.87
−
|15,948
+
|align="right " | 15,948
−
|G
+
|align="right " | G
−
|L_38eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/L_38eval.txt L_38eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|5
 
|5
 
|Ponce de Leon &amp; Inesta
 
|Ponce de Leon &amp; Inesta
−
|24.84%
+
|align="right " | 24.84%
−
|1.40
+
|align="right " | 1.40
−
|15.26%
+
|align="right " | 15.26%
−
|1.13
+
|align="right " | 1.13
−
|821
+
|align="right " | 821
−
|L
+
|align="right " | L
−
|PI_38eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/PI_38eval.txt PI_38eval.txt]
−
|
 
−
|
 
 
|----
 
|----
−
|9 Classes
+
|}
−
|
+
 
−
|
+
===9 Classes===
−
|
+
{| border="1" cellspacing="0"
−
|
+
|- style="background: yellow; text-align: center;"
−
|
+
! colspan="9" | 9 Classes
−
|
+
|-style="background: yellow;"
−
|
+
!Rank
−
|
+
!Participant
−
|
+
!Hierarchical Classification Accuracy
−
|
+
!Hierarchical Classification Accuracy Std
−
|----
+
!Raw Classification Accuracy
−
|Rank
+
!Raw Classification Accuracy Std
−
|Participant
+
!Runtime (s)
−
|Hierarchical Classification Accuracy
+
!Machine
−
|Hierarchical Classification Accuracy Std
+
!Confusion Matrix Files
−
|Raw Classification Accuracy
 
−
|Raw Classification Accuracy Std
 
−
|Runtime (s)
 
−
|Machine
 
−
|Confusion Matrix Files
 
−
|
 
−
|
 
 
|----
 
|----
 
|1
 
|1
 
|McKay &amp; Fujinaga
 
|McKay &amp; Fujinaga
−
|90.00%  
+
|align="right " | 90.00%  
−
|0.60
+
|align="right " | 0.60
−
|84.44%
+
|align="right " | 84.44%
−
|1.41
+
|align="right " | 1.41
−
|18,375
+
|align="right " | 18,375
−
|R
+
|align="right " | R
−
|MF_9eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/MF_9eval.txt MF_9eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|2
 
|2
 
|Basili, Serafini, &amp; Stellato (NB)
 
|Basili, Serafini, &amp; Stellato (NB)
−
|81.56%
+
|align="right " | 81.56%
−
|0.76
+
|align="right " | 0.76
−
|72.00%
+
|align="right " | 72.00%
−
|0.88
+
|align="right " | 0.88
−
|N/A
+
|align="right " | N/A
−
|N/A
+
|align="right " | N/A
−
|BST_NB_9eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/BST_NB_9eval.txt BST_NB_9eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|3
 
|3
 
|Li, M.  
 
|Li, M.  
−
|80.22%
+
|align="right " | 80.22%
−
|1.47
+
|align="right " | 1.47
−
|72.00%
+
|align="right " | 72.00%
−
|2.31
+
|align="right " | 2.31
−
|3,777
+
|align="right " | 3,777
−
|G
+
|align="right " | G
−
|L_9eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/L_9eval.txt L_9eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|4
 
|4
 
|Basili, Serafini, &amp; Stellato (J48)
 
|Basili, Serafini, &amp; Stellato (J48)
−
|76.67%
+
|align="right " | 76.67%
−
|1.11
+
|align="right " | 1.11
−
|65.33%
+
|align="right " | 65.33%
−
|1.65
+
|align="right " | 1.65
−
|N/A
+
|align="right " | N/A
−
|N/A
+
|align="right " | N/A
−
|BST_J48_9eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/BST_J48_9eval.txt BST_J48_9eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|5
 
|5
 
|Ponce de Leon &amp; Inesta  
 
|Ponce de Leon &amp; Inesta  
−
|50.67%
+
|align="right " | 50.67%
−
|1.26
+
|align="right " | 1.26
−
|37.78%
+
|align="right " | 37.78%
−
|2.30
+
|align="right " | 2.30
−
|197
+
|align="right " | 197
−
|L
+
|align="right " | L
−
|PI_9eval.txt
+
|[https://www.music-ir.org/mirex/results/2005/sym-genre/PI_9eval.txt PI_9eval.txt]
−
|
 
−
|
 
 
|----
 
|----
 
|}
 
|}

Latest revision as of 17:14, 2 August 2010

Introduction

Goal

To classify MIDI recordings into genre categories.

Dataset

Two sets of genre categories were used, one consisting of 38 categories and one consisting of 9 categories. Each category was represented by 25 MIDI files.Thus, the 38 genre test contained 950 MIDI files and the 9 genre test contained 225 MIDI files.Test runs were 3-fold cross validated with each algorithm tested using identical training and testing data splits.

Results

Overall

OVERALL
Rank Participant Mean Hierarchical Classification Accuracy Mean Raw Classification Accuracy
1 McKay & Fujinaga 77.17% 65.28%
2 Basili, Serafini, & Stellato (NB) 72.08% 58.53%
3 Li, M. 67.57% 55.90%
4 Basili, Serafini, & Stellato (J48) 67.14% 53.14%
5 Ponce de Leon & Inesta 37.76% 26.52%

38 Classes

38 Classes
Rank Participant Hierarchical Classification Accuracy Hierarchical Classification Accuracy Std Raw Classification Accuracy Raw Classification Accuracy Std Runtime (s) Machine Confusion Matrix Files
1 McKay & Fujinaga 64.33% 1.04 46.11% 1.51 3 days R MF_38eval.txt
2 Basili, Serafini, & Stellato (NB) 62.60% 0.26 45.05% 0.55 N/A N/A BST_NB_38eval.txt
3 Basili, Serafini, & Stellato (J48) 57.61% 1.14 40.95% 1.35 N/A N/A BST_J48_38eval.txt
4 Li, M. 54.91% 0.66 39.79% 0.87 15,948 G L_38eval.txt
5 Ponce de Leon & Inesta 24.84% 1.40 15.26% 1.13 821 L PI_38eval.txt

9 Classes

9 Classes
Rank Participant Hierarchical Classification Accuracy Hierarchical Classification Accuracy Std Raw Classification Accuracy Raw Classification Accuracy Std Runtime (s) Machine Confusion Matrix Files
1 McKay & Fujinaga 90.00% 0.60 84.44% 1.41 18,375 R MF_9eval.txt
2 Basili, Serafini, & Stellato (NB) 81.56% 0.76 72.00% 0.88 N/A N/A BST_NB_9eval.txt
3 Li, M. 80.22% 1.47 72.00% 2.31 3,777 G L_9eval.txt
4 Basili, Serafini, & Stellato (J48) 76.67% 1.11 65.33% 1.65 N/A N/A BST_J48_9eval.txt
5 Ponce de Leon & Inesta 50.67% 1.26 37.78% 2.30 197 L PI_9eval.txt
Retrieved from "https://www.music-ir.org/mirex/w/index.php?title=2005:Symbolic_Genre_Classification_Results&oldid=7619"