Difference between revisions of "2009:Audio Tag Classification (Mood Set) Results"
(→Friedman test results) |
|||
Line 57: | Line 57: | ||
<csv p=3>tag/Mood/affinity_tag_AUC_ROC.csv</csv> | <csv p=3>tag/Mood/affinity_tag_AUC_ROC.csv</csv> | ||
− | ==Friedman test | + | ==Friedman's Test Results== |
+ | ===Tag F-measure (Binary) Friedman Test=== | ||
+ | The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the F-measure for each '''tag''' in the test, averaged over all folds. | ||
+ | |||
+ | <csv p=3>tag/Mood/binary_FMeasure.friedman.tukeyKramerHSD.csv</csv> | ||
+ | |||
+ | [[Image:Binary_FMeasure.friedman.tukeyKramerHSD.png]] | ||
− | ===AUC-ROC Tag Friedman | + | |
+ | ===AUC-ROC Tag Friedman Test=== | ||
The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the Area Under the ROC curve (AUC-ROC) for each '''tag''' in the test, averaged over all folds. | The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the Area Under the ROC curve (AUC-ROC) for each '''tag''' in the test, averaged over all folds. | ||
− | <csv>tag/ | + | <csv p=3>tag/Mood/affinity.AUC_ROC_TRACK.friedman.tukeyKramerHSD.csv</csv> |
[[Image:Affinity.AUC_ROC_TAG.friedman.tukeyKramerHSD.png]] | [[Image:Affinity.AUC_ROC_TAG.friedman.tukeyKramerHSD.png]] | ||
Line 79: | Line 86: | ||
[[Image:Binary_Accuracy.friedman.tukeyKramerHSD.png]] | [[Image:Binary_Accuracy.friedman.tukeyKramerHSD.png]] | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
==Beta-Binomial test results== | ==Beta-Binomial test results== |
Revision as of 20:07, 13 October 2009
Contents
General Legend
Team ID
BP1 = Juan José Burred, Geoffroy Peeters
BP2 = Juan José Burred, Geoffroy Peeters
CL1 = Chuan Cao,Ming Li
CL2 = Chuan Cao,Ming Li
CL3 = Chuan Cao,Ming Li
CL4 = Chuan Cao,Ming Li
GP = Geoffroy Peeters
GT1 = George Tzanetakis
GT2 = George Tzanetakis
LWW1 = Hung-Yi Lo,Ju-Chiang Wang,Hsin-Min Wang
LWW2 = Hung-Yi Lo,Ju-Chiang Wang,Hsin-Min Wang
HCB = Matthew D.Hoffman,David M. Blei,Perry R.Cook
Overall Summary Results (Binary)
file /nema-raid/www/mirex/results/tag/Mood/summary_binary.csv not found
Summary Binary Relevance F-Measure (Average Across All Folds)
file /nema-raid/www/mirex/results/tag/Mood/binary_avg_Fmeasure.csv not found
Summary Binary Accuracy (Average Across All Folds)
file /nema-raid/www/mirex/results/tag/Mood/binary_avg_Accuracy.csv not found
Summary Positive Example Accuracy (Average Across All Folds)
file /nema-raid/www/mirex/results/tag/Mood/binary_avg_positive_example_Accuracy.csv not found
Summary Negative Example Accuracy (Average Across All Folds)
file /nema-raid/www/mirex/results/tag/Mood/binary_avg_negative_example_Accuracy.csv not found
file /nema-raid/www/mirex/results/tag/rounded/tag.affinity_tag_AUC_ROC.csv not found
Overall Summary Results (Affinity)
file /nema-raid/www/mirex/results/tag/Mood/summary_affinity.csv not found
Summary AUC-ROC Tag (Average Across All Folds)
file /nema-raid/www/mirex/results/tag/Mood/affinity_tag_AUC_ROC.csv not found
Friedman's Test Results
Tag F-measure (Binary) Friedman Test
The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the F-measure for each tag in the test, averaged over all folds.
file /nema-raid/www/mirex/results/tag/Mood/binary_FMeasure.friedman.tukeyKramerHSD.csv not found
File:Binary FMeasure.friedman.tukeyKramerHSD.png
AUC-ROC Tag Friedman Test
The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the Area Under the ROC curve (AUC-ROC) for each tag in the test, averaged over all folds.
file /nema-raid/www/mirex/results/tag/Mood/affinity.AUC_ROC_TRACK.friedman.tukeyKramerHSD.csv not found
File:Affinity.AUC ROC TAG.friedman.tukeyKramerHSD.png
AUC-ROC Track Friedman test
The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the Area Under the ROC curve (AUC-ROC) for each track in the test. Each track appears in exactly once over all three folds of the test. However, we are uncertain if these measurements are truly independent as an multiple tracks from each artist are used.
file /nema-raid/www/mirex/results/tag/friedmansTables/tag.affinity.AUC_ROC_TRACK.friedman.tukeyKramerHSD.csv not found
File:Affinity.AUC ROC TRACK.friedman.tukeyKramerHSD.png
Tag Classification Accuracy Friedman test
The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the classification accuracy for each tag in the test, averaged over all folds.
file /nema-raid/www/mirex/results/tag/friedmansTables/tag.binary_Accuracy.friedman.tukeyKramerHSD.csv not found
File:Binary Accuracy.friedman.tukeyKramerHSD.png
Beta-Binomial test results
Accuracy on positive examples Beta-Binomial results
The following table and plot show the results of simulations from the Beta-Binomial model using the accuracy of each algorithm's classification only on the positive examples. It only shows the relative proportion of true positives and false negatives, and should be considered with the classification accuracy on the negative examples. The image shows the estimate of the overall performance with 95% confidence intervals.
file /nema-raid/www/mirex/results/tag/tag.binary.per.fold.positive.example.accuracy.betabinomial.csv not found
File:Binary per fold positive example Accuracy.png
The plots for each tag are more interesting and the 95% confidence intervals are much tighter. Since there are so many of them, it is difficult to post them to the wiki. You can download a tar.gz zip file containing all of them here.
Accuracy on negative examples Beta-Binomial results
The following table and plot show the results of simulations from the Beta-Binomial model using the accuracy of each algorithm's classification only on the negative examples. It only shows the relative proportion of true negatives and false positives, and should be considered with the classification accuracy on the positive examples. The image shows the estimate of the overall performance with 95% confidence intervals.
file /nema-raid/www/mirex/results/tag/tag.binary.per.fold.negative.example.accuracy.betabinomial.csv not found
File:Binary per fold negative example Accuracy.png
The plots for each tag are more interesting and the 95% confidence intervals are much tighter. Since there are so many of them, it is difficult to post them to the wiki. You can download a tar.gz file containing all of them here.
Assorted Results Files for Download
AUC-ROC Clip Data
(Too large for easy Wiki viewing)
tag.affinity_clip_AUC_ROC.csv
CSV Files Without Rounding (Averaged across folds)
tag.affinity.clip.auc.roc.csv
tag.affinity.clip.auc.roc.csv
tag.binary.accuracy.csv
tag.binary.fmeasure.csv
tag.binary.negative.example.accuracy.csv
tag.binary.positive.example.accuracy.csv
CSV Files Without Rounding (Fold information)
tag.binary.per.fold.positive.example.accuracy.csv
tag.binary.per.fold.negative.example.accuracy.csv
tag.binary.per.fold.fmeasure.csv
tag.binary.per.fold.accuracy.csv
tag.affinity.tag.per.fold.auc.roc.csv
Results By Algorithm
(.tar.gz)
LB = L. Barrington, D. Turnbull, G. Lanckriet
BBE 1 = T. Bertin-Mahieux, Y. Bengio, D. Eck (KNN)
BBE 2 = T. Bertin-Mahieux, Y. Bengio, D. Eck (NNet)
BBE 3 = T. Bertin-Mahieux, D. Eck, P. Lamere, Y. Bengio (Thierry/Lamere Boosting)
TB = Bertin-Mahieux (dumb/smurf)
ME1 = M. I. Mandel, D. P. W. Ellis 1
ME2 = M. I. Mandel, D. P. W. Ellis 2
ME3 = M. I. Mandel, D. P. W. Ellis 3
GP1 = G. Peeters 1
GP2 = G. Peeters 2
TTKV = K. Trohidis, G. Tsoumakas, G. Kalliris, I. Vlahavas