Difference between revisions of "2017:Automatic Lyrics-to-Audio Alignment Results"
From MIREX Wiki
(→Results) |
|||
(10 intermediate revisions by 3 users not shown) | |||
Line 34: | Line 34: | ||
==Hansen's Dataset A cappella== | ==Hansen's Dataset A cappella== | ||
+ | |||
+ | song list: | ||
+ | * Coldplay - Clocks | ||
+ | * Katy Perry - I kissed a girl | ||
+ | * Amy Winehouse - Rehab | ||
+ | * Adele - Rolling in the deep | ||
+ | * Norah Jones - Sunrise | ||
+ | * Nickelback - This afternoon | ||
+ | * Kelis - Trick me | ||
+ | * Rihanna - Umbrella | ||
+ | * Coldplay - Viva la vida | ||
+ | |||
====Summary Results==== | ====Summary Results==== | ||
<csv>2017/ala/results_summary_acap.csv</csv> | <csv>2017/ala/results_summary_acap.csv</csv> | ||
− | ====Per- | + | ====Per-track results==== |
AK1 | AK1 | ||
Line 53: | Line 65: | ||
DMS2 | DMS2 | ||
<csv>2017/ala/results_hm_acap.csv</csv> | <csv>2017/ala/results_hm_acap.csv</csv> | ||
− | |||
− | |||
== Hansen's Dataset == | == Hansen's Dataset == | ||
Line 61: | Line 71: | ||
<csv>2017/ala/results_summary_hansen.csv</csv> | <csv>2017/ala/results_summary_hansen.csv</csv> | ||
− | ====Per- | + | ====Per-track results==== |
AK1 | AK1 | ||
<csv>2017/ala/results_htk_hansen.csv</csv> | <csv>2017/ala/results_htk_hansen.csv</csv> | ||
Line 67: | Line 77: | ||
AK2 | AK2 | ||
<csv>2017/ala/results_alignD_hansen.csv</csv> | <csv>2017/ala/results_alignD_hansen.csv</csv> | ||
+ | |||
+ | AK3 | ||
+ | <csv>2017/ala/results_alignL_hansen.csv</csv> | ||
DMS1 | DMS1 | ||
Line 73: | Line 86: | ||
DMS2 | DMS2 | ||
<csv>2017/ala/results_ss_hansen.csv</csv> | <csv>2017/ala/results_ss_hansen.csv</csv> | ||
− | |||
== Mauch's Dataset == | == Mauch's Dataset == | ||
Line 80: | Line 92: | ||
<csv>2017/ala/results_summary_mauch.csv</csv> | <csv>2017/ala/results_summary_mauch.csv</csv> | ||
− | ====Per- | + | ====Per-track results==== |
AK1 | AK1 | ||
<csv>2017/ala/results_htk_mauch.csv</csv> | <csv>2017/ala/results_htk_mauch.csv</csv> | ||
Line 86: | Line 98: | ||
AK2 | AK2 | ||
<csv>2017/ala/results_alignD_Mauch.csv</csv> | <csv>2017/ala/results_alignD_Mauch.csv</csv> | ||
+ | |||
+ | AK3 | ||
+ | <csv>2017/ala/results_alignL_mauch.csv</csv> | ||
DMS1 | DMS1 |
Latest revision as of 14:10, 25 October 2018
Contents
General Legend
Sub code | Submission name | Abstract | Contributors |
---|---|---|---|
AK1 | HMM-based Alignment | Anna Kruspe | |
AK2 | DTW-based Alignment | Anna Kruspe | |
AK3 | Levenshtein-based Alignment | Anna Kruspe | |
DMS1 | Automatic Lyrics-to-Audio Alignment with Harmonic Modeling | Georgi Dzhambazov, Xavier Serra | |
DMS2 | Automatic Lyrics-to-Audio Alignment with Source Separation | Georgi Dzhambazov, Marius Miron |
Results
Hansen's Dataset A cappella
song list:
- Coldplay - Clocks
- Katy Perry - I kissed a girl
- Amy Winehouse - Rehab
- Adele - Rolling in the deep
- Norah Jones - Sunrise
- Nickelback - This afternoon
- Kelis - Trick me
- Rihanna - Umbrella
- Coldplay - Viva la vida
Summary Results
Submission | Mean error | Median error | St. dev. error | Mean percentage | Median percentage | St. dev. percentage |
---|---|---|---|---|---|---|
AK1 | 5.11 | 0.67 | 12.88 | 0.61 | 0.70 | 0.26 |
AK2 | 9.77 | 1.82 | 22.62 | 0.46 | 0.39 | 0.27 |
AK3 | 2.87 | 0.26 | 6.74 | 0.60 | 0.63 | 0.20 |
DMS1 | 4.46 | 0.37 | 10.71 | 0.55 | 0.69 | 0.27 |
DMS2 | 4.46 | 0.37 | 10.71 | 0.55 | 0.69 | 0.27 |
Per-track results
AK1
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 41.510 | 0.008 |
i_kissed_a_girl | 0.180 | 0.721 |
rehab | 0.040 | 0.927 |
rolling_in_the_deep | 0.860 | 0.438 |
sunrise | 0.670 | 0.700 |
this_afternoon | 0.990 | 0.599 |
trick_me | 0.170 | 0.751 |
umbrella | 1.450 | 0.462 |
viva_la_vida | 0.120 | 0.842 |
AK2
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 73.560 | 0.008 |
i_kissed_a_girl | 0.270 | 0.626 |
rehab | 0.290 | 0.795 |
rolling_in_the_deep | 3.930 | 0.284 |
sunrise | 0.470 | 0.761 |
this_afternoon | 1.820 | 0.385 |
trick_me | 0.110 | 0.804 |
umbrella | 2.040 | 0.297 |
viva_la_vida | 5.470 | 0.209 |
AK3
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 21.890 | 0.152 |
i_kissed_a_girl | 0.140 | 0.704 |
rehab | 0.070 | 0.889 |
rolling_in_the_deep | 1.060 | 0.489 |
sunrise | 0.760 | 0.627 |
this_afternoon | 0.260 | 0.545 |
trick_me | 0.140 | 0.762 |
umbrella | 1.300 | 0.489 |
viva_la_vida | 0.210 | 0.742 |
DMS1
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 34.710 | 0.007 |
i_kissed_a_girl | 0.160 | 0.727 |
rehab | 0.110 | 0.876 |
rolling_in_the_deep | 1.220 | 0.350 |
sunrise | 0.360 | 0.786 |
this_afternoon | 1.320 | 0.438 |
trick_me | 0.190 | 0.720 |
umbrella | 1.720 | 0.333 |
viva_la_vida | 0.370 | 0.686 |
DMS2
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 34.710 | 0.007 |
i_kissed_a_girl | 0.160 | 0.727 |
rehab | 0.110 | 0.876 |
rolling_in_the_deep | 1.220 | 0.350 |
sunrise | 0.360 | 0.786 |
this_afternoon | 1.320 | 0.438 |
trick_me | 0.190 | 0.720 |
umbrella | 1.720 | 0.333 |
viva_la_vida | 0.370 | 0.686 |
Hansen's Dataset
Summary Results
Submission | Mean error | Median error | St. dev. error | Mean percentage | Median percentage | St. dev. percentage |
---|---|---|---|---|---|---|
AK1 | 13.96 | 7.64 | 12.73 | 0.12 | 0.06 | 0.12 |
AK2 | 27.94 | 19.58 | 31.86 | 0.10 | 0.03 | 0.13 |
AK3 | 7.34 | 4.55 | 7.71 | 0.25 | 0.28 | 0.15 |
DMS1 | 26.70 | 14.94 | 29.97 | 0.04 | 0.02 | 0.05 |
DMS2 | 10.57 | 2.02 | 23.30 | 0.34 | 0.44 | 0.18 |
Per-track results
AK1
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 42.760 | 0.002 |
i_kissed_a_girl | 7.640 | 0.052 |
rehab | 6.040 | 0.061 |
rolling_in_the_deep | 2.990 | 0.245 |
sunrise | 3.240 | 0.308 |
this_afternoon | 2.650 | 0.277 |
trick_me | 18.130 | 0.078 |
umbrella | 16.330 | 0.014 |
viva_la_vida | 25.820 | 0.009 |
AK2
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 107.720 | 0.001 |
i_kissed_a_girl | 6.590 | 0.067 |
rehab | 5.780 | 0.164 |
rolling_in_the_deep | 31.660 | 0.002 |
sunrise | 4.260 | 0.405 |
this_afternoon | 24.200 | 0.029 |
trick_me | 1.800 | 0.231 |
umbrella | 49.870 | 0.004 |
viva_la_vida | 19.580 | 0.017 |
AK3
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 22.690 | 0.014 |
i_kissed_a_girl | 0.490 | 0.445 |
rehab | 0.500 | 0.456 |
rolling_in_the_deep | 2.660 | 0.228 |
sunrise | 4.550 | 0.329 |
this_afternoon | 3.290 | 0.282 |
trick_me | 6.400 | 0.202 |
umbrella | 5.610 | 0.276 |
viva_la_vida | 19.830 | 0.006 |
DMS1
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 65.430 | 0.001 |
i_kissed_a_girl | 5.720 | 0.094 |
rehab | 5.660 | 0.060 |
rolling_in_the_deep | 19.390 | 0.005 |
sunrise | 14.940 | 0.133 |
this_afternoon | 17.310 | 0.008 |
trick_me | 9.910 | 0.018 |
umbrella | 95.470 | 0.000 |
viva_la_vida | 6.440 | 0.070 |
DMS2
Track | Average absolute error | Percentage of correct segments |
---|---|---|
clocks | 76.200 | 0.010 |
i_kissed_a_girl | 1.460 | 0.443 |
rehab | 0.550 | 0.532 |
rolling_in_the_deep | 7.530 | 0.192 |
sunrise | 2.020 | 0.501 |
this_afternoon | 0.700 | 0.481 |
trick_me | 0.390 | 0.505 |
umbrella | 2.650 | 0.189 |
viva_la_vida | 3.630 | 0.212 |
Mauch's Dataset
Summary Results
Submission | Mean error | Median error | St. dev. error | Mean percentage | Median percentage | St. dev. percentage |
---|---|---|---|---|---|---|
AK1 | 17.70 | 10.14 | 17.01 | 0.08 | 0.04 | 0.10 |
AK2 | 22.23 | 16.38 | 22.08 | 0.02 | 0.00 | 0.04 |
AK3 | 9.03 | 7.52 | 7.23 | 0.15 | 0.15 | 0.10 |
DMS1 | 14.91 | 10.14 | 15.30 | 0.04 | 0.03 | 0.04 |
DMS2 | 11.64 | 5.12 | 13.31 | 0.14 | 0.14 | 0.10 |
Per-track results
AK1
Track | Average absolute error | Percentage of correct segments |
---|---|---|
Abba.KnowingMeKnowingYou | 9.400 | 0.064 |
Bangles.EternalFlame | 38.270 | 0.007 |
Blondie.CallMe | 34.340 | 0.011 |
Duffy.WarwickAvenue | 7.200 | 0.176 |
DuranDuran.OrdinaryWorld | 12.780 | 0.020 |
FranzFerdinand.DoYouWantTo | 5.520 | 0.046 |
Martika.ToySoldiers | 20.050 | 0.001 |
Muse.GuidingLight | 66.170 | 0.002 |
OtisRedding.TheDockOfTheBay | 0.920 | 0.348 |
Queen.WeAreTheChampions | 6.430 | 0.088 |
RM-P082 | 43.130 | 0.034 |
RM-P084 | 9.520 | 0.101 |
RobertPalmer.AddictedToLove | 10.750 | 0.019 |
Santana.BlackMagicWoman | 39.870 | 0.004 |
SimonAndGarfunkel.Cecilia | 5.940 | 0.041 |
TakeThat.BackForGood | 3.090 | 0.125 |
TinaTurner.WhatsLoveGotToDoWithIt | 10.850 | 0.113 |
Toto.Africa | 4.650 | 0.299 |
U2.WithOrWithoutYou | 19.570 | 0.024 |
Zweieck.She | 5.610 | 0.174 |
AK2
Track | Average absolute error | Percentage of correct segments |
---|---|---|
Abba.KnowingMeKnowingYou | 4.250 | 0.052 |
Bangles.EternalFlame | 6.680 | 0.021 |
Blondie.CallMe | 5.980 | 0.035 |
Duffy.WarwickAvenue | 16.040 | 0.051 |
DuranDuran.OrdinaryWorld | 53.460 | 0.003 |
FranzFerdinand.DoYouWantTo | 18.410 | 0.001 |
Martika.ToySoldiers | 30.370 | 0.001 |
Muse.GuidingLight | 94.140 | 0.000 |
OtisRedding.TheDockOfTheBay | 1.380 | 0.145 |
Queen.WeAreTheChampions | 18.500 | 0.018 |
RM-P082 | 14.400 | 0.002 |
RM-P084 | 12.170 | 0.004 |
RobertPalmer.AddictedToLove | 20.840 | 0.001 |
Santana.BlackMagicWoman | 16.720 | 0.003 |
SimonAndGarfunkel.Cecilia | 7.430 | 0.004 |
TakeThat.BackForGood | 3.930 | 0.045 |
TinaTurner.WhatsLoveGotToDoWithIt | 6.860 | 0.085 |
Toto.Africa | 41.390 | 0.001 |
U2.WithOrWithoutYou | 51.360 | 0.000 |
Zweieck.She | 20.240 | 0.000 |
AK3
Track | Average absolute error | Percentage of correct segments |
---|---|---|
Abba.KnowingMeKnowingYou | 4.720 | 0.162 |
Bangles.EternalFlame | 23.560 | 0.000 |
Blondie.CallMe | 8.370 | 0.023 |
Duffy.WarwickAvenue | 2.330 | 0.256 |
DuranDuran.OrdinaryWorld | 6.800 | 0.107 |
FranzFerdinand.DoYouWantTo | 2.660 | 0.145 |
Martika.ToySoldiers | 10.320 | 0.054 |
Muse.GuidingLight | 28.460 | 0.091 |
OtisRedding.TheDockOfTheBay | 1.780 | 0.335 |
Queen.WeAreTheChampions | 6.800 | 0.134 |
RM-P082 | 9.910 | 0.153 |
RM-P084 | 8.090 | 0.151 |
RobertPalmer.AddictedToLove | 4.510 | 0.106 |
Santana.BlackMagicWoman | 15.860 | 0.080 |
SimonAndGarfunkel.Cecilia | 9.360 | 0.016 |
TakeThat.BackForGood | 0.750 | 0.305 |
TinaTurner.WhatsLoveGotToDoWithIt | 10.580 | 0.250 |
Toto.Africa | 6.960 | 0.195 |
U2.WithOrWithoutYou | 17.840 | 0.196 |
Zweieck.She | 0.940 | 0.312 |
DMS1
Track | Average absolute error | Percentage of correct segments |
---|---|---|
Abba.KnowingMeKnowingYou | 18.910 | 0.003 |
Bangles.EternalFlame | 4.010 | 0.079 |
Blondie.CallMe | 8.110 | 0.062 |
Duffy.WarwickAvenue | 3.350 | 0.154 |
DuranDuran.OrdinaryWorld | 6.250 | 0.077 |
FranzFerdinand.DoYouWantTo | 5.280 | 0.031 |
Martika.ToySoldiers | 11.150 | 0.015 |
Muse.GuidingLight | 35.620 | 0.030 |
OtisRedding.TheDockOfTheBay | 3.660 | 0.062 |
Queen.WeAreTheChampions | 9.120 | 0.020 |
RM-P082 | 12.430 | 0.009 |
RM-P084 | 15.190 | 0.022 |
RobertPalmer.AddictedToLove | 8.320 | 0.014 |
Santana.BlackMagicWoman | 21.840 | 0.013 |
SimonAndGarfunkel.Cecilia | 5.180 | 0.065 |
TakeThat.BackForGood | 4.130 | 0.031 |
TinaTurner.WhatsLoveGotToDoWithIt | 30.020 | 0.000 |
Toto.Africa | 12.420 | 0.053 |
U2.WithOrWithoutYou | 70.160 | 0.000 |
Zweieck.She | 13.050 | 0.029 |
DMS2
Track | Average absolute error | Percentage of correct segments |
---|---|---|
Abba.KnowingMeKnowingYou | 4.310 | 0.206 |
Bangles.EternalFlame | 3.080 | 0.140 |
Blondie.CallMe | 4.080 | 0.140 |
Duffy.WarwickAvenue | 9.400 | 0.210 |
DuranDuran.OrdinaryWorld | 4.320 | 0.123 |
FranzFerdinand.DoYouWantTo | 42.670 | 0.000 |
Martika.ToySoldiers | 17.600 | 0.037 |
Muse.GuidingLight | 48.800 | 0.000 |
OtisRedding.TheDockOfTheBay | 0.380 | 0.374 |
Queen.WeAreTheChampions | 5.430 | 0.219 |
RM-P082 | 12.210 | 0.049 |
RM-P084 | 1.510 | 0.242 |
RobertPalmer.AddictedToLove | 4.820 | 0.057 |
Santana.BlackMagicWoman | 16.830 | 0.056 |
SimonAndGarfunkel.Cecilia | 29.230 | 0.021 |
TakeThat.BackForGood | 1.730 | 0.172 |
TinaTurner.WhatsLoveGotToDoWithIt | 9.700 | 0.147 |
Toto.Africa | 2.390 | 0.268 |
U2.WithOrWithoutYou | 11.790 | 0.102 |
Zweieck.She | 2.480 | 0.197 |