2006:Audio Melody Extraction Results

From MIREX Wiki

Introduction

These are the results for the 2006 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2006:Audio Melody Extraction page.

The aim of the MIREX audio melody extraction evaluation is to identify the predominant melody pitch contour from polyphonic musical audio. The task consists of two parts: Voicing detection (deciding whether a particular time frame contains a "melody pitch" or not), and pitch detection (deciding the most likely melody pitch for each time frame). We structure the submission to allow these parts to be done independently, i.e. it is possible (via a negative pitch value) to guess a pitch even for frames that were being judged unvoiced. The algorithms were tested on two datasets, the MIREX2005 dataset, consisting of 25 sound files, and the ADC2004 dataset, consisting of 20 sound files, both across different music styles. Moreover, each of these datasets were split into two groups: those files in which the predominant melody is sung, and those in which the predominant melody is nonvocal.

General Legend

Team ID

dressler = Karin Dressler
ryynanen = Matti Ryynänen and Anssi Klapuri
poliner = Graham Poliner and Daniel P. W. Ellis
sutton = Christopher Sutton, Emmanuel Vincent, Mark D. Plumbley and Juan P. Bello
brossier = Paul Brossier

  • Brossier does not do voiced/unvoiced detection.
  • Sutton's algorithm is designed for sung/vocal melody extraction.

Table Headings

Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Vx d' = Voicing d-prime
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy

Overall Summary Results

MIREX 2006 Audio Melody Extraction Runtime Data

Team ID Data set Machine Run-time(seconds)
brossier M05 LIN 58
brossier ADC04 LIN 30
dressler M05 FAST 48
dressler ADC04 FAST 27
ryynanen M05 LIN/FAST 773
ryynanen ADC04 LIN/FAST 440
sutton M05 FAST 8195
sutton ADC04 FAST 5014

download these results as csv

MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - All

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
dressler 90.9% 10.5% 2.58 82.9% 84.0% 82.5%
ryynanen 84.4% 12.6% 2.16 80.6% 82.3% 77.3%
poliner 89.9% 36.3% 1.63 73.2% 76.4% 71.9%
sutton 73.2% 24.9% 1.30 62.6% 65.4% 58.2%
brossier 99.7% 88.4% 1.61 57.4% 68.7% 49.6%

download these results as csv

2006 am06 adc04 all.png

Download the Excel workbook for ADC 2004 Dataset - All.


MIREX 2006 Audio Melody Extraction Summary results- ADC 2004 Dataset - Vocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
dressler 89.8% 10.9% 2.50 77.1% 78.0% 77.3%
ryynanen 85.9% 11.5% 2.28 78.3% 79.3% 76.2%
poliner 88.4% 34.5% 1.59 65.4% 69.0% 64.7%
sutton 90.8% 32.0% 1.79 67.5% 68.0% 64.2%
brossier 99.8% 93.9% 1.28 56.3% 63.5% 46.7%

download these results as csv

2006 am06 adc04 vocal.png

Download the Excel workbook for ADC 2004 Dataset - Vocal.


MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - Nonvocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
dressler 92.0% 9.5% 2.71 88.7% 90.1% 87.7%
ryynanen 82.9% 15.2% 1.98 82.8% 85.3% 78.4%
poliner 91.4% 40.4% 1.61 81.0% 83.9% 79.1%
sutton 54.6% 8.1% 1.52 57.7% 62.9% 52.3%
brossier 99.7% 82.9% 1.83 58.5% 73.8% 52.5%

download these results as csv

2006 am06 adc04 nonvocal.png

Download the Excel workbook for ADC 2004 Dataset - Nonvocal.


MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
dressler 89.3% 28.8% 1.80 77.7% 82.0% 73.2%
ryynanen 78.2% 16.5% 1.75 71.5% 75.0% 67.9%
poliner 93.5% 45.1% 1.64 66.2% 70.4% 63.0%
sutton 64.5% 13.8% 1.46 56.4% 60.1% 53.7%
brossier 99.5% 98.2% 0.46 41.0% 56.1% 31.9%

download these results as csv

2006 am06 mirex05 all.png

Download the Excel workbook for MIREX 2005 Dataset - All.


MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Vocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
dressler 85.5% 28.7% 1.62 78.5% 81.6% 73.7%
ryynanen 77.0% 15.6% 1.75 75.7% 76.9% 72.5%
poliner 93.7% 44.3% 1.68 69.1% 70.6% 65.0%
sutton 71.8% 12.3% 1.74 70.7% 71.6% 67.3%
brossier 99.6% 97.9% 0.63 42.7% 53.5% 30.7%

download these results as csv

2006 am06 mirex05 vocal.png

Download the Excel workbook for MIREX 2005 Dataset - Vocal.


MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Nonvocal

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
dressler 93.1% 30.3% 2.00 76.9% 83.1% 72.8%
ryynanen 79.3% 21.0% 1.62 64.2% 71.6% 59.6%
poliner 93.4% 49.0% 1.53 61.2% 70.1% 59.3%
sutton 57.5% 21.2% 0.99 30.8% 39.7% 29.6%
brossier 99.2% 98.8% 0.16 37.8% 60.8% 34.1%

download these results as csv

2006 am06 mirex05 nonvocal.png

Download the Excel workbook for MIREX 2005 Dataset - Nonvocal.