Difference between revisions of "2005:Audio Artist Identification Results"
From MIREX Wiki
| Line 23: | Line 23: | ||
! Rank !! Participant !! Mean of Magnatune Raw Classification Accuracy and USPOP Raw Classification Accuracy | ! Rank !! Participant !! Mean of Magnatune Raw Classification Accuracy and USPOP Raw Classification Accuracy | ||
|- | |- | ||
| − | | 1 || Mandel & Ellis || 72.45% | + | | 1 ||[https://www.music-ir.org/mirex/abstracts/2005/mandel.pdf Mandel & Ellis] || 72.45% |
|- | |- | ||
| − | |2 || Bergstra, Casagrande, & Eck (1) || 68.57% | + | |2 || [https://www.music-ir.org/mirex/abstracts/2005/bergstra.pdf Bergstra, Casagrande, & Eck (1)] || 68.57% |
|- | |- | ||
| − | | 3 || Bergstra, Casagrande, & Eck (2) || 66.71% | + | | 3 ||[https://www.music-ir.org/mirex/abstracts/2005/bergstra Bergstra, Casagrande, & Eck (2)] || 66.71% |
|- | |- | ||
| − | | 4 || Pampalk, E. || 61.28% | + | | 4 ||[https://www.music-ir.org/mirex/abstracts/2005/pampalk.pdf Pampalk, E.] || 61.28% |
|- | |- | ||
| 5 || West & Lamere || 47.24% | | 5 || West & Lamere || 47.24% | ||
|- | |- | ||
| − | | 6 || Tzanetakis, G. || 42.05% | + | | 6 ||[https://www.music-ir.org/mirex/abstracts/2005/tzanetakis.pdf Tzanetakis, G.] || 42.05% |
|- | |- | ||
| − | | 7 || Logan, B || 25.95% | + | | 7 ||[https://www.music-ir.org/mirex/abstracts/2005/logan.pdf Logan, B] || 25.95% |
|- | |- | ||
|} | |} | ||
| Line 46: | Line 46: | ||
! Rank !! Participant !! Raw Classification Accuracy !! Normalized Raw lassification Accuracy !! Runtime (s) !! Machine !! Confusion Matrix Files | ! Rank !! Participant !! Raw Classification Accuracy !! Normalized Raw lassification Accuracy !! Runtime (s) !! Machine !! Confusion Matrix Files | ||
|- | |- | ||
| − | | 1 || Bergstra, Casagrande, & Eck (1) || 77.26% || 79.64% || 24 hours || B0 || BCE_1_MTeval.txt | + | | 1 || Bergstra, Casagrande, & Eck (1) || 77.26% || 79.64% || 24 hours || B0 ||[https://www.music-ir.org/mirex/results/2005/BCE_1_MTeval.txt BCE_1_MTeval.txt] |
|- | |- | ||
| − | | 2 || Mandel & Ellis || 76.60% || 76.62% || 11073 || R || ME_MTeval.txt | + | | 2 || Mandel & Ellis || 76.60% || 76.62% || 11073 || R || [https://www.music-ir.org/mirex/results/2005/ME_MTeval.txt ME_MTeval.txt] |
|- | |- | ||
| − | | 3 || Bergstra, Casagrande, & Eck (2) || 74.45% || 74.51% || -- || -- || BCE_2_MTeval.txt | + | | 3 || Bergstra, Casagrande, & Eck (2) || 74.45% || 74.51% || -- || -- || [https://www.music-ir.org/mirex/results/2005/BCE_2_MTeval.txt BCE_2_MTeval.txt] |
|- | |- | ||
| − | | 4 || Pampalk, E. || 66.36% || 66.48% || 4272 || B1 || P_MTeval.txt | + | | 4 || Pampalk, E. || 66.36% || 66.48% || 4272 || B1 || [https://www.music-ir.org/mirex/results/2005/P_MTeval.txt P_MTeval.txt] |
|- | |- | ||
| − | | 5 || Tzanetakis, G. || 55.45% || 55.59% || 2632 || B0 || T_MTeval.txt | + | | 5 || Tzanetakis, G. || 55.45% || 55.59% || 2632 || B0 || [https://www.music-ir.org/mirex/results/2005/T_MTeval.txt T_MTeval.txt] |
|- | |- | ||
| − | | 6 || West & Lamere || 53.43% || 53.48% || 27480 || B3 || WL_MTeval.txt | + | | 6 || West & Lamere || 53.43% || 53.48% || 27480 || B3 || [https://www.music-ir.org/mirex/results/2005/WL_MTeval.txt WL_MTeval.txt] |
|- | |- | ||
| − | | 7 || Logan, B || 37.07% || 37.10% || N/A || B3 || L_MTeval.txt | + | | 7 || Logan, B || 37.07% || 37.10% || N/A || B3 || [https://www.music-ir.org/mirex/results/2005/L_MTeval.txt L_MTeval.txt] |
|- | |- | ||
| 8 || Lidy & Rauber (SSD+RH) || TO * || -- || -- || -- || -- || | | 8 || Lidy & Rauber (SSD+RH) || TO * || -- || -- || -- || -- || | ||
| Line 75: | Line 75: | ||
! Rank !! Participant !! Raw Classification Accuracy !! Normalized Raw lassification Accuracy !! Runtime (s) !! Machine !! Confusion Matrix Files | ! Rank !! Participant !! Raw Classification Accuracy !! Normalized Raw lassification Accuracy !! Runtime (s) !! Machine !! Confusion Matrix Files | ||
|- | |- | ||
| − | | 1 || Mandel & Ellis || 68.30% || 67.96% || 10240 || R || ME_USeval.txt | + | | 1 || Mandel & Ellis || 68.30% || 67.96% || 10240 || R || [https://www.music-ir.org/mirex/results/2005/ME_USeval.txt ME_USeval.txt] |
|- | |- | ||
| − | | 2 || Bergstra, Casagrande, & Eck (1) || 59.88% || 60.90% || 24 Hours || B0 || BCE_1_USeval.txt | + | | 2 || Bergstra, Casagrande, & Eck (1) || 59.88% || 60.90% || 24 Hours || B0 || [https://www.music-ir.org/mirex/results/2005/BCE_1_USeval.txt ME_USeval.txt] |
|- | |- | ||
| − | | 3 || Bergstra, Casagrande, & Eck (2) || 58.96% || 58.96% || -- || -- || BCE_2_USeval.txt | + | | 3 || Bergstra, Casagrande, & Eck (2) || 58.96% || 58.96% || -- || -- || [https://www.music-ir.org/mirex/results/2005/BCE_2_USeval.txt BCE_2_USeval.txt] |
|- | |- | ||
| − | | 4 || Pampalk, E. || 56.20% || 56.03% || 4321 || B1 || P_USeval.txt | + | | 4 || Pampalk, E. || 56.20% || 56.03% || 4321 || B1 || [https://www.music-ir.org/mirex/results/2005/P_USeval.txt P_USeval.txt] |
|- | |- | ||
| − | | 5 || West & Lamere || 41.04% || 41.00% || 26871 || B3 || WL_USeval.txt | + | | 5 || West & Lamere || 41.04% || 41.00% || 26871 || B3 || [https://www.music-ir.org/mirex/results/2005/WL_USeval.txt WL_USeval.txt] |
|- | |- | ||
| − | | 6 || Tzanetakis, G. || 28.64% || 28.48% || 2443 || B0 || T_USeval.txt | + | | 6 || Tzanetakis, G. || 28.64% || 28.48% || 2443 || B0 || [https://www.music-ir.org/mirex/results/2005/T_USeval.txt T_USeval.txt] |
|- | |- | ||
| − | | 7 || Logan, B. || 14.83% || 14.76% || N/A || B3 || L_USeval.txt | + | | 7 || Logan, B. || 14.83% || 14.76% || N/A || B3 || [https://www.music-ir.org/mirex/results/2005/L_USeval.txt L_USeval.txt] |
|- | |- | ||
| 8 || Lidy & Rauber (SSD+RH) || TO * || -- || -- || -- || -- || | | 8 || Lidy & Rauber (SSD+RH) || TO * || -- || -- || -- || -- || | ||
Revision as of 23:58, 29 July 2010
Goal: To identify artist from music audio (in PCM format).
Dataset: Two sets of data were used: Magnatune and USPOP. The audio sampling rates used were either 44.1 KHz or 22.05 KHz (mono). More data information is in the following table.
| Dataset | Size (@ 44.1 KHz) | Number of Training Files | Number of Testing Files |
|---|---|---|---|
| Magnatune | 35.2 GB | 1158 | 642 |
| USPOP | 37.3 GB | 1158 | 653 |
| OVERALL | ||
|---|---|---|
| Rank | Participant | Mean of Magnatune Raw Classification Accuracy and USPOP Raw Classification Accuracy |
| 1 | Mandel & Ellis | 72.45% |
| 2 | Bergstra, Casagrande, & Eck (1) | 68.57% |
| 3 | Bergstra, Casagrande, & Eck (2) | 66.71% |
| 4 | Pampalk, E. | 61.28% |
| 5 | West & Lamere | 47.24% |
| 6 | Tzanetakis, G. | 42.05% |
| 7 | Logan, B | 25.95% |
| Magnatune Dataset | |||||||
|---|---|---|---|---|---|---|---|
| Rank | Participant | Raw Classification Accuracy | Normalized Raw lassification Accuracy | Runtime (s) | Machine | Confusion Matrix Files | |
| 1 | Bergstra, Casagrande, & Eck (1) | 77.26% | 79.64% | 24 hours | B0 | BCE_1_MTeval.txt | |
| 2 | Mandel & Ellis | 76.60% | 76.62% | 11073 | R | ME_MTeval.txt | |
| 3 | Bergstra, Casagrande, & Eck (2) | 74.45% | 74.51% | -- | -- | BCE_2_MTeval.txt | |
| 4 | Pampalk, E. | 66.36% | 66.48% | 4272 | B1 | P_MTeval.txt | |
| 5 | Tzanetakis, G. | 55.45% | 55.59% | 2632 | B0 | T_MTeval.txt | |
| 6 | West & Lamere | 53.43% | 53.48% | 27480 | B3 | WL_MTeval.txt | |
| 7 | Logan, B | 37.07% | 37.10% | N/A | B3 | L_MTeval.txt | |
| 8 | Lidy & Rauber (SSD+RH) | TO * | -- | -- | -- | -- | |
| 8 | Lidy & Rauber (RP+SSD) | TO * | -- | -- | -- | -- | |
| 8 | Lidy & Rauber (RP+SSD+RH) | TO * | -- | -- | -- | -- | |
| USPOP Dataset | |||||||
|---|---|---|---|---|---|---|---|
| Rank | Participant | Raw Classification Accuracy | Normalized Raw lassification Accuracy | Runtime (s) | Machine | Confusion Matrix Files | |
| 1 | Mandel & Ellis | 68.30% | 67.96% | 10240 | R | ME_USeval.txt | |
| 2 | Bergstra, Casagrande, & Eck (1) | 59.88% | 60.90% | 24 Hours | B0 | ME_USeval.txt | |
| 3 | Bergstra, Casagrande, & Eck (2) | 58.96% | 58.96% | -- | -- | BCE_2_USeval.txt | |
| 4 | Pampalk, E. | 56.20% | 56.03% | 4321 | B1 | P_USeval.txt | |
| 5 | West & Lamere | 41.04% | 41.00% | 26871 | B3 | WL_USeval.txt | |
| 6 | Tzanetakis, G. | 28.64% | 28.48% | 2443 | B0 | T_USeval.txt | |
| 7 | Logan, B. | 14.83% | 14.76% | N/A | B3 | L_USeval.txt | |
| 8 | Lidy & Rauber (SSD+RH) | TO * | -- | -- | -- | -- | |
| 8 | Lidy & Rauber (RP+SSD) | TO * | -- | -- | -- | -- | |
| 8 | Lidy & Rauber (RP+SSD+RH) | TO * | -- | -- | -- | -- | |
Note:
DNC: did not complete ( error in execution).
TO: timed out (did not complete within 24 hours).
