2009:Audio Tag Classification (Mood Set) Results

From MIREX Wiki
Revision as of 18:20, 16 October 2009 by Xiaohu (talk | contribs) (Introduction)

Introduction

These are the results for the 2009 running of the Audio Tag Classification (Mood Set) task. For background information about this task set please refer to the [Audio_Tag_Classification] page. The data was created by Xiao Hu and consists of 3,469 unique songs and 135 mood tags organized into 18 mood tag groups.

Mood tags

The tags were collected from last.fm. All tags in this set are mood related as identified and grouped by WordNet-Affect and human experts.

Each mood tag group contains the following tags:

   * G12: calm, comfort, quiet, serene, mellow, chill out, calm down, calming, chillout, comforting, content, cool down, mellow music,  mellow rock, peace of mind, quietness, relaxation, serenity, solace, soothe, soothing, still, tranquil, tranquility, tranquility
   * G15: sad, sadness, unhappy, melancholic, melancholy, feeling sad, mood: sad ΓÇô slightly, sad song
   * G5: happy, happiness, happy songs, happy music, glad, mood: happy
   * G32: romantic, romantic music
   * G2: upbeat, gleeful, high spirits, zest, enthusiastic, buoyancy, elation, mood: upbeat
   * G16: depressed, blue, dark, depressive, dreary, gloom, darkness, depress, depression, depressing, gloomy
   * G28: anger, angry, choleric, fury, outraged, rage, angry music
   * G17: grief, heartbreak, mournful, sorrow, sorry, doleful, heartache, heartbreaking, heartsick, lachrymose, mourning, plaintive, regret, sorrowful
   * G14: dreamy
   * G6: cheerful, cheer up, festive, jolly, jovial, merry, cheer, cheering, cheery, get happy, rejoice, songs that are cheerful, sunny
   * G8: brooding, contemplative, meditative, reflective, broody, pensive, pondering, wistful
   * G29: aggression, aggressive
   * G25: angst, anxiety, anxious, jumpy, nervous, angsty
   * G9: confident, encouraging,  encouragement, optimism, optimistic
   * G7: desire, hope, hopeful, mood: hopeful
   * G11: earnest, heartfelt
   * G31: pessimism, cynical, pessimistic, weltschmerz, cynical/sarcastic
   * G1: excitement, exciting, exhilarating, thrill, ardor, stimulating, thrilling, titillating

For details on the mood tag groups, please see

X. Hu, J. S. Downie, A.Ehmann (2009). Lyric Text Mining in Music Mood Classification, In the 10th International Symposium on Music Information Retrieval (ISMIR 2009), Oct. 2009, Kobe, Japan

Data

The songs are Western pop songs mostly from the USPOP collection. Each song may belong to multiple mood tag groups. The main rationale on songs selection is: if more than one tag in a group were applied to a song, or if one tag in a group was applied more than once to a song, this song is marked as belonging to this group.

For details on how the songs were selected, please see the Mood multi-tag data description.

Audio format: 30 second clips, 44.1kHz, stereo,16bit, WAV files; The data were split into 3 folds with artist filtering.




General Legend

Team ID

BP1 = Juan José Burred, Geoffroy Peeters
BP2 = Juan José Burred, Geoffroy Peeters
CC1 = Chuan Cao, Ming Li
CC2 = Chuan Cao, Ming Li
CC3 = Chuan Cao, Ming Li
CC4 = Chuan Cao, Ming Li
GP = Geoffroy Peeters
GT1 = George Tzanetakis
GT2 = George Tzanetakis
HCB = Matthew D.Hoffman, David M. Blei, Perry R.Cook
LWW1 = Hung-Yi Lo, Ju-Chiang Wang, Hsin-Min Wang
LWW2 = Hung-Yi Lo, Ju-Chiang Wang, Hsin-Min Wang

Overall Summary Results (Binary)

file /nema-raid/www/mirex/results/tag/Mood/summary_binary.csv not found


Summary Binary Relevance F-Measure (Average Across All Folds)

file /nema-raid/www/mirex/results/tag/Mood/binary_avg_Fmeasure.csv not found

Summary Binary Accuracy (Average Across All Folds)

file /nema-raid/www/mirex/results/tag/Mood/binary_avg_Accuracy.csv not found

Summary Positive Example Accuracy (Average Across All Folds)

file /nema-raid/www/mirex/results/tag/Mood/binary_avg_positive_example_Accuracy.csv not found

Summary Negative Example Accuracy (Average Across All Folds)

file /nema-raid/www/mirex/results/tag/Mood/binary_avg_negative_example_Accuracy.csv not found

Overall Summary Results (Affinity)

file /nema-raid/www/mirex/results/tag/Mood/summary_affinity.csv not found

Summary AUC-ROC Tag (Average Across All Folds)

file /nema-raid/www/mirex/results/tag/Mood/affinity_tag_AUC_ROC.csv not found

Select Friedman's Test Results

Tag F-measure (Binary) Friedman Test

The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the F-measure for each tag in the test, averaged over all folds.

file /nema-raid/www/mirex/results/tag/Mood/binary_FMeasure.friedman.tukeyKramerHSD.csv not found


https://music-ir.org/mirex/2009/results/tag/Mood/small.binary_FMeasure.friedman.tukeyKramerHSD.png

Per Track F-measure (Binary) Friedman Test

The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the F-measure for each track in the test, averaged over all folds. file /nema-raid/www/mirex/results/tag/Mood/binary_FMeasure_per_track.friedman.tukeyKramerHSD.csv not found


https://music-ir.org/mirex/2009/results/tag/Mood/small.binary_FMeasure_per_track.friedman.tukeyKramerHSD.png

Tag AUC-ROC (Affinity) Friedman Test

The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the Area Under the ROC curve (AUC-ROC) for each tag in the test, averaged over all folds.

file /nema-raid/www/mirex/results/tag/Mood/affinity.AUC_ROC_TAG.friedman.tukeyKramerHSD.csv not found

https://music-ir.org/mirex/2009/results/tag/Mood/small.affinity.AUC_ROC_TAG.friedman.tukeyKramerHSD.png


Per Track AUC-ROC (Affinity) Friedman Test

The following table and plot show the results of Friedman's ANOVA with Tukey-Kramer multiple comparisons computed over the Area Under the ROC curve (AUC-ROC) for each track/clip in the test, averaged over all folds.

file /nema-raid/www/mirex/results/tag/Mood/affinity.AUC_ROC_TRACK.friedman.tukeyKramerHSD.csv not found

https://music-ir.org/mirex/2009/results/tag/Mood/small.affinity.AUC_ROC_TRACK.friedman.tukeyKramerHSD.png

Assorted Results Files for Download

General Results

affinity_tag_fold_AUC_ROC.csv
affinity_clip_AUC_ROC.csv
binary_per_fold_Accuracy.csv
binary_per_fold_Fmeasure.csv
binary_per_fold_negative_example_Accuracy.csv
binary_per_fold_per_track_Accuracy.csv
binary_per_fold_per_track_Fmeasure.csv
binary_per_fold_per_track_negative_example_Accuracy.csv
binary_per_fold_per_track_positive_example_Accuracy.csv
binary_per_fold_positive_example_Accuracy.csv

Friedman's Tests Results

affinity.PrecisionAt3.friedman.tukeyKramerHSD.csv
affinity.PrecisionAt3.friedman.tukeyKramerHSD.png
affinity.PrecisionAt6.friedman.tukeyKramerHSD.csv
affinity.PrecisionAt6.friedman.tukeyKramerHSD.png
affinity.PrecisionAt9.friedman.tukeyKramerHSD.csv
affinity.PrecisionAt9.friedman.tukeyKramerHSD.png
affinity.PrecisionAt12.friedman.tukeyKramerHSD.csv
affinity.PrecisionAt12.friedman.tukeyKramerHSD.png
affinity.PrecisionAt15.friedman.tukeyKramerHSD.csv
affinity.PrecisionAt15.friedman.tukeyKramerHSD.png
binary_Accuracy.friedman.tukeyKramerHSD.csv
binary_Accuracy.friedman.tukeyKramerHSD.png

Results By Algorithm

(.tgz format)

BP1 = Juan José Burred, Geoffroy Peeters
BP2 = Juan José Burred, Geoffroy Peeters
CC1 = Chuan Cao, Ming Li
CC2 = Chuan Cao, Ming Li
CC3 = Chuan Cao, Ming Li
CC4 = Chuan Cao, Ming Li
GP = Geoffroy Peeters
GT1 = George Tzanetakis
GT2 = George Tzanetakis
LWW1 = Hung-Yi Lo, Ju-Chiang Wang, Hsin-Min Wang
LWW2 = Hung-Yi Lo, Ju-Chiang Wang, Hsin-Min Wang
HCB = Matthew D.Hoffman, David M. Blei, Perry R.Cook