2007:Audio Genre Classification Results

From MIREX Wiki
Revision as of 20:23, 13 May 2010 by IMIRSELBot (talk | contribs) (Robot: Automated text replacement (-\[http://www.music-ir.org/mirex/(200[5-9])/abs/ +[http://www.music-ir.org/mirex/abs/\1/))

Introduction

These are the results for the 2007 running of the Audio Genre Classification task. For background information about this task set please refer to the 2007:Audio Genre Classification page. The data set consisted of 7000 30 second clips covering the following genres (700 tracks per genre):

A  : BAROQUE
B  : BLUES
C  : CLASSICAL
D  : COUNTRY
E  : EDANCE
F  : JAZZ
G  : METAL
H  : RAPHIPHOP
I  : ROCKROLL
J  : ROMANTIC

Partial scores were earned for some confusions (i.e., ROCKROLL-METAL; JAZZ-BLUES, etc.)

General Legend

Team ID

ME = Michael I. Mandel, Daniel P. W. Ellis
TL = Thomas Lidy, Andreas Rauber, Antonio Pertusa, José Manuel Iñesta
GT = George Tzanetakis
GH = Enric Guaus, Perfecto Herrera
IM = IMIRSEL M2K

Overall Summary Results

MIREX 2007 Audio Genre Classification Summary Results - Raw and Hierarchical Classification Accuracy Averaged Over Three Train/Test Folds

Participant Avg. Hierarchical Classification Accuracy Avg. Raw Classification Accuracy
GH 71.87% 62.89%
IM_knn 64.83% 54.87%
IM_svm 76.56% 68.29%
TL 75.57% 66.71%
ME 75.03% 66.60%
ME_spec 73.57% 65.50%
GT 74.15% 65.34%

download these results as csv

MIREX 2007 Audio Genre Classification Evaluation Logs and Confusion Matrices

GH
IM_knn
IM_svm
TL
ME
ME_spec
GT

MIREX 2007 Audio Genre Classification Run Times

Participant Runtime (sec) / Fold
GH Feat Ex: 22740 Train/Classify: 194
IM-knn Feat Ex: 6879 Train/Classify: 1245
IM-svm Feat Ex: 6879 Train/Classify: 51
TL Feat Ex: 54192 Train/Classify: 147
ME Feat Ex: 8166 Train/Classify: 207
ME_spec Feat Ex: 8018 Train/Classify: 210
GT Feat Ex/Train/Classify: 1442

download these results as csv