Difference between revisions of "2008:Audio Melody Extraction Results"

From MIREX Wiki
(→‎Overall Summary Results)
(→‎Team ID)
 
(25 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
==Introduction==
 
==Introduction==
−
These are the results for the 2008 running of the Audio Melody Extraction task set. For background information about this task set please refer to the [[Audio Melody Extraction]] page.
+
These are the results for the 2008 running of the Audio Melody Extraction task set. For background information about this task set please refer to the [[2008:Audio Melody Extraction]] page. Special thanks to Jean-Louis Durrieu for doing the vocal/non-vocal split summaries.
  
 
===General Legend===
 
===General Legend===
 
====Team ID====  
 
====Team ID====  
  
−
'''PC''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf P. Cancela]<br />
+
'''PC''' = [https://www.music-ir.org/mirex/abstracts/2008/AudioMelodyExt_pcancela.pdf P. Cancela]<br />
−
'''CLLY1''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf  C. Cao, M. Li, J. Liu, Y. Yan 1]<br />
+
'''CLLY1''' = [https://www.music-ir.org/mirex/abstracts/2008/mirex08_multiF0_CC.pdf  C. Cao, M. Li, J. Liu, Y. Yan 1]<br />
−
'''CLLY2''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf  C. Cao, M. Li, J. Liu, Y. Yan 2]<br />
+
'''CLLY2''' = [https://www.music-ir.org/mirex/abstracts/2008/mirex08_multiF0_CC.pdf  C. Cao, M. Li, J. Liu, Y. Yan 2]<br />
−
'''DRD1''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf  J-L. Durrieu, G. Richard, B. David 1]<br />
+
'''DRD1''' = [https://www.music-ir.org/mirex/abstracts/2008/durrieu_imm_gmm.pdf  J-L. Durrieu, G. Richard, B. David 1]<br />
−
'''DRD2''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf  J-L. Durrieu, G. Richard, B. David 2]<br />
+
'''DRD2''' = [https://www.music-ir.org/mirex/abstracts/2008/durrieu_imm_gmm.pdf  J-L. Durrieu, G. Richard, B. David 2]<br />
−
'''RK''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf  M. Ryyn├ñnen, A. Klapuri]<br />
+
'''RK''' = [https://www.music-ir.org/mirex/abstracts/2008/ME_ryynanen.pdf  M. Ryynänen, A. Klapuri]<br />
−
'''VR''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf  V. Rao]<br />
+
'''VR''' = [https://www.music-ir.org/mirex/abstracts/2008/ME_rao.pdf  V. Rao, P. Rao]<br />
  
 
====Table Headings====
 
====Table Headings====
Line 23: Line 23:
 
==Overall Summary Results==
 
==Overall Summary Results==
  
−
===MIREX 2008 Audio Melody Extraction Runtime Data===
+
===MIREX 2008 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets - All===
−
<csv>am08_runtime.csv</csv>
+
<csv>2008/am08_overall.csv</csv>
  
 
===MIREX 2008 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All===
 
===MIREX 2008 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All===
−
<csv>am08_m08_all.csv</csv>
+
<csv>2008/am08_m08_all.csv</csv>
 +
 
 +
[[Image:2008_am08_m08_all.png]]
 +
 
 +
Download the [https://www.music-ir.org/mirex/results/2008/am08_persong_m08_all.xls Excel workbook] for MIREX 2008 Dataset - All (NB: it seems that all the songs of this dataset are Vocal ones, there is therefore no vocal/non-vocal separate results).
 +
 
 +
===MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal===
 +
<csv>2008/am08_m05_vocal.csv</csv>
 +
 
 +
[[Image:2008_am08_m05_vocal.png]]
 +
 
 +
Download the [https://www.music-ir.org/mirex/results/2008/am08_persong_m05_vocal.xls Excel Workbook] for MIREX 2005 Dataset - vocal.
 +
 
 +
===MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal===
 +
<csv>2008/am08_m05_nonvocal.csv</csv>
  
−
Download the [ Excel workbook] for MIREX 2008 Dataset - All.
+
[[Image:2008_am08_m05_nonvocal.png]]
  
−
[[Image:Am08_m08_all.png]]
+
Download the [https://www.music-ir.org/mirex/results/2008/am08_persong_m05_nonvocal.xls Excel Workbook] for MIREX 2005 Dataset - nonvocal.
  
 
===MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All===
 
===MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All===
−
<csv>am08_m05_all.csv</csv>
+
<csv>2008/am08_m05_all.csv</csv>
 +
 
 +
[[Image:2008_am08_m05_all2.png]]
 +
 
 +
Download the [https://www.music-ir.org/mirex/results/2008/am08_persong_m05_all.xls Excel Workbook] for MIREX 2005 Dataset - All.
 +
 
 +
===MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal===
 +
<csv>2008/am08_adc04_vocal.csv</csv>
  
−
[[Image:Am08_m05_all.png]]
+
[[Image:2008_am08_adc04_vocal.png]]
  
−
Download the [ Excel workbook] for MIREX 2005 Dataset - All.
+
Download the [https://www.music-ir.org/mirex/results/2008/am08_persong_adc04_vocal.xls Excel Workbook] for ADC 2004 Dataset - vocal.
  
 +
===MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal===
 +
<csv>2008/am08_adc04_nonvocal.csv</csv>
 +
 +
[[Image:2008_am08_adc04_nonvocal.png]]
 +
 +
Download the [https://www.music-ir.org/mirex/results/2008/am08_persong_adc04_nonvocal.xls Excel Workbook] for ADC 2004 Dataset - nonvocal.
  
 
===MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - All===
 
===MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - All===
−
<csv>am08_adc04_all.csv.csv</csv>
+
<csv>2008/am08_adc04_all.csv</csv>
  
−
Download the [ Excel workbook] for ADC 2004 Dataset - All.
+
[[Image:2008_am08_adc04_all.png]]
−
 
 
−
[[Image:Am08_adc04_all.png]]
 
  
 +
Download the [https://www.music-ir.org/mirex/results/2008/am08_persong_adc04_all.xls Excel workbook] for ADC 2004 Dataset - All.
  
 +
===MIREX 2008 Audio Melody Extraction Runtime Data===
 +
<csv>2008/am08_runtime.csv</csv>
 
[[Category: Results]]
 
[[Category: Results]]

Latest revision as of 17:14, 23 July 2010

Introduction

These are the results for the 2008 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2008:Audio Melody Extraction page. Special thanks to Jean-Louis Durrieu for doing the vocal/non-vocal split summaries.

General Legend

Team ID

PC = P. Cancela
CLLY1 = C. Cao, M. Li, J. Liu, Y. Yan 1
CLLY2 = C. Cao, M. Li, J. Liu, Y. Yan 2
DRD1 = J-L. Durrieu, G. Richard, B. David 1
DRD2 = J-L. Durrieu, G. Richard, B. David 2
RK = M. Ryynänen, A. Klapuri
VR = V. Rao, P. Rao

Table Headings

Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Vx d' = Voicing d-prime
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy

Overall Summary Results

MIREX 2008 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets - All

Avg. Overall Acc.
clly1 49.8%
clly2 62.1%
drd1 58.6%
drd2 73.2%
pc 76.1%
rk 71.1%
vr 67.1%

download these results as csv

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
drd1 98.7% 68.9% 1.74 85.8% 88.3% 76.0%
rk 90.4% 48.5% 1.34 83.5% 83.8% 75.3%
drd2 96.6% 57.1% 1.65 81.8% 82.6% 75.0%
pc 95.7% 65.4% 1.32 83.9% 84.0% 73.3%
vr 68.3% 21.1% 1.28 88.2% 88.6% 66.7%
clly1 77.6% 43.4% 0.92 54.7% 55.3% 51.4%
clly2 86.1% 67.8% 0.62 54.7% 55.3% 49.7%

download these results as csv

2008 am08 m08 all.png

Download the Excel workbook for MIREX 2008 Dataset - All (NB: it seems that all the songs of this dataset are Vocal ones, there is therefore no vocal/non-vocal separate results).

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 83.1% 19.7% 1.81 77.6% 78.1% 76.8%
vr 81.5% 20.5% 1.72 76.1% 79.2% 71.8%
rk 89.5% 45.0% 1.38 74.6% 76.6% 67.6%
drd2 92.6% 51.1% 1.42 75.0% 77.9% 66.7%
clly2 81.5% 46.0% 1.00 70.1% 72.8% 63.7%
clly1 75.1% 37.5% 1.00 70.1% 72.8% 62.7%
drd1 91.7% 63.6% 1.04 54.7% 62.4% 48.0%

download these results as csv

2008 am08 m05 vocal.png

Download the Excel Workbook for MIREX 2005 Dataset - vocal.

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
drd2 90.4% 39.7% 1.56 67.2% 72.6% 64.1%
drd1 88.1% 44.4% 1.32 60.1% 68.3% 57.7%
pc 80.9% 21.1% 1.68 58.3% 62.4% 56.5%
clly2 71.1% 50.0% 0.56 64.6% 68.3% 55.1%
rk 78.4% 41.5% 1.00 63.4% 68.7% 54.8%
vr 84.7% 40.7% 1.26 56.9% 69.9% 51.5%
clly1 27.8% 50.5% -0.60 64.6% 68.3% 24.1%

download these results as csv

2008 am08 m05 nonvocal.png

Download the Excel Workbook for MIREX 2005 Dataset - nonvocal.

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 83.9% 19.9% 1.83 71.0% 72.8% 69.8%
drd2 92.0% 49.2% 1.42 72.4% 76.2% 66.0%
vr 85.1% 23.8% 1.75 69.7% 76.5% 64.9%
rk 85.6% 44.4% 1.21 71.2% 74.4% 63.5%
clly2 77.8% 46.7% 0.85 68.9% 72.0% 61.4%
drd1 91.9% 60.4% 1.14 57.4% 65.3% 52.2%
clly1 58.2% 42.2% 0.40 68.9% 72.0% 48.9%

download these results as csv

2008 am08 m05 all2.png

Download the Excel Workbook for MIREX 2005 Dataset - All.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 91.4% 11.0% 2.60 89.5% 89.8% 88.3%
drd2 92.7% 30.9% 1.95 87.4% 87.6% 83.2%
rk 89.2% 29.3% 1.78 84.0% 84.2% 79.9%
vr 80.2% 20.9% 1.66 81.6% 88.1% 72.0%
drd1 95.4% 59.8% 1.44 77.2% 83.4% 69.6%
clly2 87.3% 53.1% 1.06 71.8% 72.3% 64.9%
clly1 62.5% 32.2% 0.78 71.8% 72.3% 52.3%

download these results as csv

2008 am08 adc04 vocal.png

Download the Excel Workbook for ADC 2004 Dataset - vocal.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 82.6% 6.9% 2.42 78.5% 80.1% 80.2%
drd2 90.2% 33.2% 1.72 83.3% 84.1% 79.0%
rk 88.4% 18.9% 2.07 80.2% 82.6% 77.2%
clly2 80.2% 39.7% 1.11 80.4% 83.4% 72.7%
vr 85.7% 17.7% 1.99 70.4% 80.8% 67.2%
clly1 45.4% 26.1% 0.53 80.4% 83.4% 47.2%
drd1 91.3% 53.5% 1.27 49.0% 62.5% 44.6%

download these results as csv

2008 am08 adc04 nonvocal.png

Download the Excel Workbook for ADC 2004 Dataset - nonvocal.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - All

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 88.0% 9.8% 2.47 85.1% 85.9% 85.1%
drd2 91.7% 31.6% 1.86 85.7% 86.2% 81.5%
rk 88.9% 26.2% 1.86 82.4% 83.5% 78.8%
vr 82.3% 20.0% 1.77 77.1% 85.2% 70.1%
clly2 84.6% 49.1% 1.04 75.3% 76.7% 68.0%
drd1 93.8% 57.9% 1.34 65.9% 75.0% 59.6%
clly1 55.7% 29.7% 0.67 75.3% 76.7% 50.2%

download these results as csv

2008 am08 adc04 all.png

Download the Excel workbook for ADC 2004 Dataset - All.

MIREX 2008 Audio Melody Extraction Runtime Data

Team ID Data set Machine Run-time(seconds)
clly1 M05 FAST3 56
clly1 ADC04 FAST3 36
clly1 M08 FAST3 49
clly2 M05 FAST3 57
clly2 ADC04 FAST3 39
clly2 M08 FAST3 50
drd1 M05 ALE 3 days
drd1 ADC04 ALE 5 days
drd1 M08 ALE 4 days
drd2 M05 ALE 3294
drd2 ADC04 ALE 2082
drd2 M08 ALE 2318
pc M05 FAST 61470
pc ADC04 FAST 33974
pc M08 FAST 47031
rk M05 ALE 115
rk ADC04 ALE 73
rk M08 ALE 100
vr M05 FAST 200
vr ADC04 FAST 110
vr M08 FAST 184

download these results as csv

Retrieved from "https://www.music-ir.org/mirex/w/index.php?title=2008:Audio_Melody_Extraction_Results&oldid=7386"