University of Illinois Graduate School of Library and Information Science ISRL
NOTE: As of October 6, 2005, there are still 1 set of evaluations to be completed.
Goal: To identify artist from music audio (in PCM format).
Dataset: Two sets
of data
were used: Magnatune and USPOP.
The audio sampling rates used were either 44.1 KHz or 22.05 KHz (mono).
More data information is in the following table.
| Dataset | Size
(@ 44.1 KHz) |
Number of Training Files | Number of Testing Files |
| Magnatune | 35.2 GB | 1158 | 642 |
| USPOP | 37.3 GB | 1158 |
653 |
|
|
||
| Rank | Participant | Mean
of Magnatune Raw Classification Accuracy and USPOP Raw Classification Accuracy |
| 1 | Mandel & Ellis | 72.45% |
| 2 | Bergstra, Casagrande, & Eck (1) | 68.57% |
| 3 |
Bergstra, Casagrande, & Eck (2) | 66.71% |
| 4 |
Pampalk, E. | 61.28% |
| 5 | West & Lamere | 47.24% |
| 6 |
Tzanetakis, G. | 42.05% |
| 7 |
Logan, B | 25.95% |
|
|
||||||||
| Rank | Participant | Raw Classification Accuracy | Normalized Raw lassification Accuracy | Runtime (s) | Machine | Confusion
Matrix Files |
||
| 1 |
Bergstra, Casagrande, & Eck (1) | 77.26% |
79.64% |
24
hours |
B0 |
BCE_1_MTeval.txt | ||
| 2 |
Mandel & Ellis | 76.60% | 76.62% | 11073 | R | ME_MTeval.txt | ||
| 3 |
Bergstra, Casagrande, & Eck (2) | 74.45% |
74.51% |
BCE_2_MTeval.txt | ||||
| 4 |
Pampalk, E. | 66.36% |
66.48% |
4272 |
B1 |
P_MTeval.txt | ||
| 5 |
Tzanetakis, G. | 55.45% |
55.59% |
2632 |
B0 |
T_MTeval.txt | ||
| 6 |
West & Lamere |
53.43% |
53.48% |
27480 |
B3 |
WL_MTeval.txt | ||
| 7 |
Logan, B |
37.07% |
37.10% |
N/A |
B3 |
L_MTeval.txt | ||
| 8 |
Lidy & Rauber (SSD+RH) | TO
* |
||||||
| 8 |
Lidy & Rauber (RP+SSD) | TO
* |
||||||
| 8 |
Lidy & Rauber (RP+SSD+RH) | TO
* |
||||||
|
|
||||||
| Rank | Participant | Raw Classification Accuracy | Normalized Raw Classification Accuracy | Runtime (s) | Machine | Confusion
Matrix Files |
| 1 |
Mandel & Ellis |
68.30% |
67.96% |
10240 |
R |
ME_USeval.txt |
| 2 |
Bergstra,
Casagrande, & Eck (1) |
59.88% |
60.90% |
24
Hours |
B0 |
BCE_1_USeval.txt |
| 3 |
Bergstra, Casagrande, & Eck (2) | 58.96% |
58.96% |
BCE_2_USeval.txt | ||
| 4 |
Pampalk, E. |
56.20% |
56.03% |
4321 |
B1 |
P_USeval.txt |
| 5 |
West &
Lamere |
41.04% |
41.00% |
26871 |
B3 |
WL_USeval.txt |
| 6 |
Tzanetakis, G. | 28.64% | 28.48% | 2443 | B0 | T_USeval.txt |
| 7 |
Logan, B. |
14.83% |
14.76% |
N/A |
B3 |
L_USeval.txt |
| 8 |
Lidy & Rauber (SSD+RH) | TO * |
||||
| 8 |
Lidy & Rauber (RP+SSD) | TO
* |
||||
| 8 |
Lidy & Rauber (RP+SSD+RH) | TO * |
||||