Difference between revisions of "2018:Automatic Lyrics-to-Audio Alignment Results"

From MIREX Wiki
(→‎Results)
(→‎Hansen's dataset mixed)
Line 67: Line 67:
  
  
−
===Hansen's dataset mixed===
+
===Hansen's dataset mix===
 
====Summary Results====
 
====Summary Results====
−
<csv>2018/ala/hansen/mixed/summary_hansen_mixed.csv</csv>
+
<csv>2018/ala/hansen/mix/summary_hansen_mix.csv</csv>
  
 
====Per-track results====
 
====Per-track results====
  
 
CW2
 
CW2
−
<csv>2018/ala/hansen/mixed/CW2_hansen_mixed.csv</csv>
+
<csv>2018/ala/hansen/mix/CW2_hansen_mix.csv</csv>
  
 
CW3
 
CW3
−
<csv>2018/ala/hansen/mixed/CW3_hansen_mixed.csv</csv>
+
<csv>2018/ala/hansen/mix/CW3_hansen_mix.csv</csv>
  
 
GSLW1
 
GSLW1
−
<csv>2018/ala/hansen/mixed/GSLW1_hansen_mixed.csv</csv>
+
<csv>2018/ala/hansen/mix/GSLW1_hansen_mix.csv</csv>
  
 
GSLW2
 
GSLW2
−
<csv>2018/ala/hansen/mixed/GSLW2_hansen_mixed.csv</csv>
+
<csv>2018/ala/hansen/mix/GSLW2_hansen_mix.csv</csv>
  
 
GSLW3
 
GSLW3
−
<csv>2018/ala/hansen/mixed/GSLW3_hansen_mixed.csv</csv>
+
<csv>2018/ala/hansen/mix/GSLW3_hansen_mix.csv</csv>
 +
 
 
===Mauch's dataset===
 
===Mauch's dataset===
 
===Gracenote dataset===
 
===Gracenote dataset===

Revision as of 09:37, 4 November 2018

General Legend

Sub code Submission name Abstract Contributors
CW1 HMM-based Alignment subtask1 [PDF] Chung-Che Wang
CW2 HMM-based Alignment monophone acoustic models subtask2 PDF Chung-Che Wang
CW3 HMM-based Alignment bi-phone acoustic models subtask2 PDF Chung-Che Wang
GSLW1 GMM-HMM (SAT) Models subtask2 PDF Chitralekha Gupta, Bidisha Sharma, Haizhou Li, Ye Wang
GSLW2 SAT+DNN Models subtask2 PDF Chitralekha Gupta, Bidisha Sharma, Haizhou Li, Ye Wang
GSLW3 Non-vocal suppression and SAT Models subtask 2 PDF Chitralekha Gupta, Bidisha Sharma, Haizhou Li, Ye Wang
FFJ1 subtask 1 & 2 [PDF] Yang-Bin Dell Fan, Zhe-Cheng Fan, Jyh-Shing Roger Jang

Results

Subtask1: A cappella Mandarin Chinese pop songs

Source-separated

Mixed

Subtask2: English pop songs

Hansen's dataset a cappella

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW2 0.35 0.65 0.79
CW3 0.47 0.62 0.81
GSLW1 2.66 0.39 0.48
GSLW2 2.90 0.35 0.40
GSLW3 5.80 0.31 0.31

download these results as csv

Per-track results

CW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.31 0.570093890373057 0.777372262773723
i_kissed_a_girl 0.2 0.703915182929027 0.848484848484848
clocks_cut 0.19 0.790849995302038 0.835294117647059
rehab 0.17 0.805194335473958 0.865131578947369
rolling_in_the_deep 0.82 0.481665469290785 0.680555555555556
sunrise 0.52 0.513409547058522 0.672
this_afternoon 0.18 0.68360542512406 0.895555555555555
trick_me 0.5 0.530746028428777 0.772413793103448
umbrella 0.52 0.546212118854114 0.72
viva_la_vida 0.13 0.842945012210246 0.879432624113475

download these results as csv

CW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.32 0.545755317403897 0.744525547445255
i_kissed_a_girl 0.25 0.674245470835476 0.837121212121212
clocks_cut 0.15 0.767154471687807 0.894117647058824
rehab 0.13 0.796814864279409 0.898026315789474
rolling_in_the_deep 0.57 0.522259080218718 0.727777777777778
sunrise 1.72 0.267711468622025 0.504
this_afternoon 0.15 0.683714791923987 0.915555555555556
trick_me 0.86 0.562015684740021 0.875862068965517
umbrella 0.4 0.616793375576461 0.8425
viva_la_vida 0.17 0.78921921137274 0.875886524822695

download these results as csv

GSLW1

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.27 0.543259128122919 0.805860805860806
i_kissed_a_girl 0.11 0.763207570129874 0.946969696969697
clocks_cut 0.63 0.554917222456744 0.905882352941176
rehab 2.07 0.071219687000756 0.05921052631579
rolling_in_the_deep 3.15 0.009252640391624 0.027777777777778
sunrise 3.68 0.127399217474657 0.08
this_afternoon 13.61 0 0
trick_me 2.97 0.080867269977835 0
umbrella 0.08 0.795244553082193 0.97323600973236
viva_la_vida 0.04 0.926735844657678 0.98936170212766

download these results as csv

GSLW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.48 0.47449184033124 0.684981684981685
i_kissed_a_girl 0.34 0.647207424371483 0.829545454545455
clocks_cut 2.33 0.570135233602421 0.717647058823529
rehab 0.85 0.144820599809062 0.006578947368421
rolling_in_the_deep 3.19 0.016695413804996 0.013888888888889
sunrise 4.44 0.041273457518474 0.008
this_afternoon 13.66 0.006072269630639 0
trick_me 2.98 0.058593676899486 0.006896551724138
umbrella 0.48 0.788270711351441 0.788321167883212
viva_la_vida 0.2 0.766924833336262 0.968085106382979

download these results as csv

GSLW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 19.85 0.002186924613191 0
i_kissed_a_girl 7.61 0.026256826598641 0.035294117647059
clocks_cut 0.85 0.144820599809062 0.006578947368421
rehab 9.5 0.010902142506288 0.013888888888889
rolling_in_the_deep 4 0.186314997894924 0
sunrise 13.06 0.028738675715158 0.057777777777778
this_afternoon 2.97 0.080867269977835 0
trick_me 0.08 0.795244553082193 0.97323600973236
umbrella 0.04 0.926735844657678 0.98936170212766
viva_la_vida 0.04 0.926735844658 0.99

download these results as csv


Hansen's dataset mix

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW2 3.566 0.37802036168513 0.567124924061636
CW3 2.07 0.449797259112808 0.699532109376408
GSLW1 7.01 0.25 0.43
GSLW2 13.32 0.09 0.16
GSLW3 17.78 0.02 0.02

download these results as csv

Per-track results

CW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 10.99 0.226124698582149 0.416058394160584
clocks_cut 3.26 0.38930810261648 0.740740740740741
i_kissed_a_girl 0.27 0.621119507879228 0.772727272727273
rehab 0.62 0.530159948664057 0.61437908496732
rolling_in_the_deep 0.69 0.475431465929193 0.694444444444444
sunrise 1.61 0.320656199160953 0.456
this_afternoon 2.31 0.407519185676804 0.61283185840708
trick_me 6.22 0.190157715996446 0.351724137931035
umbrella 8.55 0.179398573857059 0.288939051918736
viva_la_vida 1.14 0.440328218488935 0.723404255319149

download these results as csv

CW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 8.2 0.311763886339647 0.58029197080292
clocks_cut 1.94 0.41005479556285 0.802469135802469
i_kissed_a_girl 0.28 0.649341636045947 0.867424242424242
rehab 0.24 0.637445516497089 0.797385620915033
rolling_in_the_deep 0.48 0.500342720534644 0.733333333333333
sunrise 1.04 0.379106004445562 0.68
this_afternoon 2.22 0.469125484189428 0.694690265486726
trick_me 2.38 0.27435310698658 0.520689655172414
umbrella 1.83 0.444926153029708 0.652370203160271
viva_la_vida 2.09 0.421513287496619 0.666666666666667

download these results as csv

GSLW1

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 14.68 0.141693609781414 0.283088235294118
clocks_cut 2.55 0.414053417584349 0.790123456790123
i_kissed_a_girl 0.58 0.503784408057355 0.727272727272727
rehab 1.76 0.27223584836753 0.42156862745098
rolling_in_the_deep 1.82 0.414947059854735 0.619444444444444
sunrise 2.57 0.289407480532988 0.56
this_afternoon 7.4 0.020940117418009 0.019911504424779
trick_me 2.08 0.329192639487664 0.675862068965517
umbrella 6.18 0.001593684168864 0
viva_la_vida 30.47 0.065665086051737 0.163120567375887

download these results as csv

GSLW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 28.59 0.036920580858665 0.040441176470588
clocks_cut 16.57 0.187578305002503 0.185185185185185
i_kissed_a_girl 13.51 0.111778055591569 0.181818181818182
rehab 3.67 0.110359041696525 0.225490196078431
rolling_in_the_deep 16.8 0.05627968058309 0.069444444444445
sunrise 3.25 0.214156673851977 0.384
this_afternoon 7.41 0.020940117418009 0.019911504424779
trick_me 4.15 0.177744909245817 0.448275862068966
umbrella 10.44 0.012630023232327 0.029197080291971
viva_la_vida 28.79 0.017583635407709 0.024822695035461

download these results as csv

GSLW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 29.65 0 0
clocks_cut 7.7 0.077433469687045 0.037037037037037
i_kissed_a_girl 19.85 0.002186924613191 0
rehab 6.5 0.060910986687061 0.058823529411765
rolling_in_the_deep 12.37 0.010704204247576 0.011111111111111
sunrise 31.52 0 0
this_afternoon 7.41 0.020940117418009 0.019911504424779
trick_me 44.46 0.002999541552481 0
umbrella 6.62 0 0
viva_la_vida 11.72 0.029197955750477 0.060283687943263

download these results as csv

Mauch's dataset

Gracenote dataset

Retrieved from "https://www.music-ir.org/mirex/w/index.php?title=2018:Automatic_Lyrics-to-Audio_Alignment_Results&oldid=12866"