Difference between revisions of "2017:Automatic Lyrics-to-Audio Alignment Results"

From MIREX Wiki
(→‎Results)
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
−
== General Legend ==
+
= General Legend =
 
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
 
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
 
     |- style="background: yellow"
 
     |- style="background: yellow"
Line 30: Line 30:
  
 
|}
 
|}
−
 
  
 
=Results=
 
=Results=
  
 
==Hansen's Dataset A cappella==
 
==Hansen's Dataset A cappella==
 +
 +
song list:
 +
* Coldplay - Clocks
 +
* Katy Perry - I kissed a girl
 +
* Amy Winehouse - Rehab
 +
* Adele - Rolling in the deep
 +
* Norah Jones - Sunrise
 +
* Nickelback - This afternoon
 +
* Kelis - Trick me
 +
* Rihanna - Umbrella
 +
* Coldplay - Viva la vida
 +
 
====Summary Results====
 
====Summary Results====
 
<csv>2017/ala/results_summary_acap.csv</csv>
 
<csv>2017/ala/results_summary_acap.csv</csv>
  
−
====Per-song results====
+
====Per-track results====
  
 
AK1
 
AK1
Line 55: Line 66:
 
<csv>2017/ala/results_hm_acap.csv</csv>
 
<csv>2017/ala/results_hm_acap.csv</csv>
  
−
 
+
== Hansen's Dataset ==
−
 
 
−
===Hansen's Dataset===
 
  
 
====Summary Results====
 
====Summary Results====
 
<csv>2017/ala/results_summary_hansen.csv</csv>
 
<csv>2017/ala/results_summary_hansen.csv</csv>
  
−
====Per-song results====
+
====Per-track results====
 
AK1
 
AK1
 
<csv>2017/ala/results_htk_hansen.csv</csv>
 
<csv>2017/ala/results_htk_hansen.csv</csv>
Line 68: Line 77:
 
AK2
 
AK2
 
<csv>2017/ala/results_alignD_hansen.csv</csv>
 
<csv>2017/ala/results_alignD_hansen.csv</csv>
 +
 +
AK3
 +
<csv>2017/ala/results_alignL_hansen.csv</csv>
  
 
DMS1
 
DMS1
Line 75: Line 87:
 
<csv>2017/ala/results_ss_hansen.csv</csv>
 
<csv>2017/ala/results_ss_hansen.csv</csv>
  
−
 
+
== Mauch's Dataset ==
−
===Mauch's Dataset===
 
  
 
====Summary Results====
 
====Summary Results====
 
<csv>2017/ala/results_summary_mauch.csv</csv>
 
<csv>2017/ala/results_summary_mauch.csv</csv>
  
−
====Per-song results====
+
====Per-track results====
 
AK1
 
AK1
 
<csv>2017/ala/results_htk_mauch.csv</csv>
 
<csv>2017/ala/results_htk_mauch.csv</csv>
Line 87: Line 98:
 
AK2
 
AK2
 
<csv>2017/ala/results_alignD_Mauch.csv</csv>
 
<csv>2017/ala/results_alignD_Mauch.csv</csv>
 +
 +
AK3
 +
<csv>2017/ala/results_alignL_mauch.csv</csv>
  
 
DMS1
 
DMS1

Latest revision as of 14:10, 25 October 2018

General Legend

Sub code Submission name Abstract Contributors
AK1 HMM-based Alignment PDF Anna Kruspe
AK2 DTW-based Alignment PDF Anna Kruspe
AK3 Levenshtein-based Alignment PDF Anna Kruspe
DMS1 Automatic Lyrics-to-Audio Alignment with Harmonic Modeling PDF Georgi Dzhambazov, Xavier Serra
DMS2 Automatic Lyrics-to-Audio Alignment with Source Separation PDF Georgi Dzhambazov, Marius Miron

Results

Hansen's Dataset A cappella

song list:

  • Coldplay - Clocks
  • Katy Perry - I kissed a girl
  • Amy Winehouse - Rehab
  • Adele - Rolling in the deep
  • Norah Jones - Sunrise
  • Nickelback - This afternoon
  • Kelis - Trick me
  • Rihanna - Umbrella
  • Coldplay - Viva la vida

Summary Results

Submission Mean error Median error St. dev. error Mean percentage Median percentage St. dev. percentage
AK1 5.11 0.67 12.88 0.61 0.70 0.26
AK2 9.77 1.82 22.62 0.46 0.39 0.27
AK3 2.87 0.26 6.74 0.60 0.63 0.20
DMS1 4.46 0.37 10.71 0.55 0.69 0.27
DMS2 4.46 0.37 10.71 0.55 0.69 0.27

download these results as csv

Per-track results

AK1

Track Average absolute error Percentage of correct segments
clocks 41.510 0.008
i_kissed_a_girl 0.180 0.721
rehab 0.040 0.927
rolling_in_the_deep 0.860 0.438
sunrise 0.670 0.700
this_afternoon 0.990 0.599
trick_me 0.170 0.751
umbrella 1.450 0.462
viva_la_vida 0.120 0.842

download these results as csv

AK2

Track Average absolute error Percentage of correct segments
clocks 73.560 0.008
i_kissed_a_girl 0.270 0.626
rehab 0.290 0.795
rolling_in_the_deep 3.930 0.284
sunrise 0.470 0.761
this_afternoon 1.820 0.385
trick_me 0.110 0.804
umbrella 2.040 0.297
viva_la_vida 5.470 0.209

download these results as csv

AK3

Track Average absolute error Percentage of correct segments
clocks 21.890 0.152
i_kissed_a_girl 0.140 0.704
rehab 0.070 0.889
rolling_in_the_deep 1.060 0.489
sunrise 0.760 0.627
this_afternoon 0.260 0.545
trick_me 0.140 0.762
umbrella 1.300 0.489
viva_la_vida 0.210 0.742

download these results as csv

DMS1

Track Average absolute error Percentage of correct segments
clocks 34.710 0.007
i_kissed_a_girl 0.160 0.727
rehab 0.110 0.876
rolling_in_the_deep 1.220 0.350
sunrise 0.360 0.786
this_afternoon 1.320 0.438
trick_me 0.190 0.720
umbrella 1.720 0.333
viva_la_vida 0.370 0.686

download these results as csv

DMS2

Track Average absolute error Percentage of correct segments
clocks 34.710 0.007
i_kissed_a_girl 0.160 0.727
rehab 0.110 0.876
rolling_in_the_deep 1.220 0.350
sunrise 0.360 0.786
this_afternoon 1.320 0.438
trick_me 0.190 0.720
umbrella 1.720 0.333
viva_la_vida 0.370 0.686

download these results as csv

Hansen's Dataset

Summary Results

Submission Mean error Median error St. dev. error Mean percentage Median percentage St. dev. percentage
AK1 13.96 7.64 12.73 0.12 0.06 0.12
AK2 27.94 19.58 31.86 0.10 0.03 0.13
AK3 7.34 4.55 7.71 0.25 0.28 0.15
DMS1 26.70 14.94 29.97 0.04 0.02 0.05
DMS2 10.57 2.02 23.30 0.34 0.44 0.18

download these results as csv

Per-track results

AK1

Track Average absolute error Percentage of correct segments
clocks 42.760 0.002
i_kissed_a_girl 7.640 0.052
rehab 6.040 0.061
rolling_in_the_deep 2.990 0.245
sunrise 3.240 0.308
this_afternoon 2.650 0.277
trick_me 18.130 0.078
umbrella 16.330 0.014
viva_la_vida 25.820 0.009

download these results as csv

AK2

Track Average absolute error Percentage of correct segments
clocks 107.720 0.001
i_kissed_a_girl 6.590 0.067
rehab 5.780 0.164
rolling_in_the_deep 31.660 0.002
sunrise 4.260 0.405
this_afternoon 24.200 0.029
trick_me 1.800 0.231
umbrella 49.870 0.004
viva_la_vida 19.580 0.017

download these results as csv

AK3

Track Average absolute error Percentage of correct segments
clocks 22.690 0.014
i_kissed_a_girl 0.490 0.445
rehab 0.500 0.456
rolling_in_the_deep 2.660 0.228
sunrise 4.550 0.329
this_afternoon 3.290 0.282
trick_me 6.400 0.202
umbrella 5.610 0.276
viva_la_vida 19.830 0.006

download these results as csv

DMS1

Track Average absolute error Percentage of correct segments
clocks 65.430 0.001
i_kissed_a_girl 5.720 0.094
rehab 5.660 0.060
rolling_in_the_deep 19.390 0.005
sunrise 14.940 0.133
this_afternoon 17.310 0.008
trick_me 9.910 0.018
umbrella 95.470 0.000
viva_la_vida 6.440 0.070

download these results as csv

DMS2

Track Average absolute error Percentage of correct segments
clocks 76.200 0.010
i_kissed_a_girl 1.460 0.443
rehab 0.550 0.532
rolling_in_the_deep 7.530 0.192
sunrise 2.020 0.501
this_afternoon 0.700 0.481
trick_me 0.390 0.505
umbrella 2.650 0.189
viva_la_vida 3.630 0.212

download these results as csv

Mauch's Dataset

Summary Results

Submission Mean error Median error St. dev. error Mean percentage Median percentage St. dev. percentage
AK1 17.70 10.14 17.01 0.08 0.04 0.10
AK2 22.23 16.38 22.08 0.02 0.00 0.04
AK3 9.03 7.52 7.23 0.15 0.15 0.10
DMS1 14.91 10.14 15.30 0.04 0.03 0.04
DMS2 11.64 5.12 13.31 0.14 0.14 0.10

download these results as csv

Per-track results

AK1

Track Average absolute error Percentage of correct segments
Abba.KnowingMeKnowingYou 9.400 0.064
Bangles.EternalFlame 38.270 0.007
Blondie.CallMe 34.340 0.011
Duffy.WarwickAvenue 7.200 0.176
DuranDuran.OrdinaryWorld 12.780 0.020
FranzFerdinand.DoYouWantTo 5.520 0.046
Martika.ToySoldiers 20.050 0.001
Muse.GuidingLight 66.170 0.002
OtisRedding.TheDockOfTheBay 0.920 0.348
Queen.WeAreTheChampions 6.430 0.088
RM-P082 43.130 0.034
RM-P084 9.520 0.101
RobertPalmer.AddictedToLove 10.750 0.019
Santana.BlackMagicWoman 39.870 0.004
SimonAndGarfunkel.Cecilia 5.940 0.041
TakeThat.BackForGood 3.090 0.125
TinaTurner.WhatsLoveGotToDoWithIt 10.850 0.113
Toto.Africa 4.650 0.299
U2.WithOrWithoutYou 19.570 0.024
Zweieck.She 5.610 0.174

download these results as csv

AK2

Track Average absolute error Percentage of correct segments
Abba.KnowingMeKnowingYou 4.250 0.052
Bangles.EternalFlame 6.680 0.021
Blondie.CallMe 5.980 0.035
Duffy.WarwickAvenue 16.040 0.051
DuranDuran.OrdinaryWorld 53.460 0.003
FranzFerdinand.DoYouWantTo 18.410 0.001
Martika.ToySoldiers 30.370 0.001
Muse.GuidingLight 94.140 0.000
OtisRedding.TheDockOfTheBay 1.380 0.145
Queen.WeAreTheChampions 18.500 0.018
RM-P082 14.400 0.002
RM-P084 12.170 0.004
RobertPalmer.AddictedToLove 20.840 0.001
Santana.BlackMagicWoman 16.720 0.003
SimonAndGarfunkel.Cecilia 7.430 0.004
TakeThat.BackForGood 3.930 0.045
TinaTurner.WhatsLoveGotToDoWithIt 6.860 0.085
Toto.Africa 41.390 0.001
U2.WithOrWithoutYou 51.360 0.000
Zweieck.She 20.240 0.000

download these results as csv

AK3

Track Average absolute error Percentage of correct segments
Abba.KnowingMeKnowingYou 4.720 0.162
Bangles.EternalFlame 23.560 0.000
Blondie.CallMe 8.370 0.023
Duffy.WarwickAvenue 2.330 0.256
DuranDuran.OrdinaryWorld 6.800 0.107
FranzFerdinand.DoYouWantTo 2.660 0.145
Martika.ToySoldiers 10.320 0.054
Muse.GuidingLight 28.460 0.091
OtisRedding.TheDockOfTheBay 1.780 0.335
Queen.WeAreTheChampions 6.800 0.134
RM-P082 9.910 0.153
RM-P084 8.090 0.151
RobertPalmer.AddictedToLove 4.510 0.106
Santana.BlackMagicWoman 15.860 0.080
SimonAndGarfunkel.Cecilia 9.360 0.016
TakeThat.BackForGood 0.750 0.305
TinaTurner.WhatsLoveGotToDoWithIt 10.580 0.250
Toto.Africa 6.960 0.195
U2.WithOrWithoutYou 17.840 0.196
Zweieck.She 0.940 0.312

download these results as csv

DMS1

Track Average absolute error Percentage of correct segments
Abba.KnowingMeKnowingYou 18.910 0.003
Bangles.EternalFlame 4.010 0.079
Blondie.CallMe 8.110 0.062
Duffy.WarwickAvenue 3.350 0.154
DuranDuran.OrdinaryWorld 6.250 0.077
FranzFerdinand.DoYouWantTo 5.280 0.031
Martika.ToySoldiers 11.150 0.015
Muse.GuidingLight 35.620 0.030
OtisRedding.TheDockOfTheBay 3.660 0.062
Queen.WeAreTheChampions 9.120 0.020
RM-P082 12.430 0.009
RM-P084 15.190 0.022
RobertPalmer.AddictedToLove 8.320 0.014
Santana.BlackMagicWoman 21.840 0.013
SimonAndGarfunkel.Cecilia 5.180 0.065
TakeThat.BackForGood 4.130 0.031
TinaTurner.WhatsLoveGotToDoWithIt 30.020 0.000
Toto.Africa 12.420 0.053
U2.WithOrWithoutYou 70.160 0.000
Zweieck.She 13.050 0.029

download these results as csv

DMS2

Track Average absolute error Percentage of correct segments
Abba.KnowingMeKnowingYou 4.310 0.206
Bangles.EternalFlame 3.080 0.140
Blondie.CallMe 4.080 0.140
Duffy.WarwickAvenue 9.400 0.210
DuranDuran.OrdinaryWorld 4.320 0.123
FranzFerdinand.DoYouWantTo 42.670 0.000
Martika.ToySoldiers 17.600 0.037
Muse.GuidingLight 48.800 0.000
OtisRedding.TheDockOfTheBay 0.380 0.374
Queen.WeAreTheChampions 5.430 0.219
RM-P082 12.210 0.049
RM-P084 1.510 0.242
RobertPalmer.AddictedToLove 4.820 0.057
Santana.BlackMagicWoman 16.830 0.056
SimonAndGarfunkel.Cecilia 29.230 0.021
TakeThat.BackForGood 1.730 0.172
TinaTurner.WhatsLoveGotToDoWithIt 9.700 0.147
Toto.Africa 2.390 0.268
U2.WithOrWithoutYou 11.790 0.102
Zweieck.She 2.480 0.197

download these results as csv

Retrieved from "https://www.music-ir.org/mirex/w/index.php?title=2017:Automatic_Lyrics-to-Audio_Alignment_Results&oldid=12858"