2007:Multiple Fundamental Frequency Estimation & Tracking Results

From MIREX Wiki
Revision as of 19:28, 13 May 2010 by IMIRSELBot (talk | contribs) (Robot: Automated text replacement (-[[Image: +[[Image:2007_))

Introduction

These are the results for the 2007 running of the Multiple Fundamental Frequency Estimation and Tracking task. For background information about this task set please refer to the 2007:Multiple Fundamental Frequency Estimation & Tracking page.


General Legend

Team ID

CC1 = Chuan Cao, Ming Li, Jian Liu, Yonghong Yan 1
CC2 = Chuan Cao, Ming Li, Jian Liu, Yonghong Yan 2
AC1 = Arshia Cont 1
AC2 = Arshia Cont 2
AC3 = Arshia Cont 3
AC4 = Arshia Cont 4
KE1 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama 1
KE2 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama 2
KE3 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama 3
KE4 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama 4
VE1 = Valentin Emiya, Roland Badeau, Bertrand David 1
VE2 = Valentin Emiya, Roland Badeau, Bertrand David 2
PL = Pierre Leveau
PI1 = Antonio Pertusa, José Manuel Iñesta 1
PI2 = Antonio Pertusa, José Manuel Iñesta 2
PI3 = Antonio Pertusa, José Manuel Iñesta 3
PE1 = Graham Poliner, Daniel P. W. Ellis 1
PE2 = Graham Poliner, Daniel P. W. Ellis 2
SR = Stanisław A. Raczyński, Nobutaka Ono, Shigeki Sagayama
RK = Matt Ryynänen, Anssi Klapuri
EV1 = Emmanuel Vincent, Nancy Bertin, Roland Badeau 1
EV2 = Emmanuel Vincent, Nancy Bertin, Roland Badeau 2
EV3 = Emmanuel Vincent, Nancy Bertin, Roland Badeau 3
EV4 = Emmanuel Vincent, Nancy Bertin, Roland Badeau 4
CY = Chunghsin Yeh
ZR = Ruohua Zhou, Joshua D. Reiss

Overall Summary Results Task1

Below are the average scores across 28 test files. These files consisted of 7 groups, each group having 4 files ranging from 2 polyphony to 5 polyphony. 20 real recordings, 8 synthesized from RWC samples.

file /nema-raid/www/mirex/results/multiF0Task1.results.csv not found

Where

File:2007 Ev formulas.PNG

  • Nref is the number of non-zero elements in the ground truth data.
  • Nsys is the number of active elements returned by the system.
  • Ncorr is the number of correctly identified elements.

Individual Results Files for Task 1

Individual results: Scores per Query

AC1 = Arshia Cont
AC2 = Arshia Cont
CC1 = Chuan Cao, Ming Li, Jian Liu, Yonghong Yan 1
CC2 = Chuan Cao, Ming Li, Jian Liu, Yonghong Yan 1
CY = Chunghsin Yeh
EV1 = Emmanuel Vincent, Nancy Bertin, Roland Badeau
EV2 = Emmanuel Vincent, Nancy Bertin, Roland Badeau
KE1 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama
KE2 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama
PE = Graham Poliner, Daniel P. W. Ellis
PI1 = Antonio Pertusa, José Manuel Iñesta 1
PL = Pierre Leveau
RK = Matt Ryynänen, Anssi Klapuri
SR = Stanislaw A. Raczynski, Nobutaka Ono, Shigeki Sagayama
VE1 = Valentin Emiya, Roland Badeau, Bertrand David
ZR = JRuohua Zhou, Joshua D. Reiss

Info about the filenames

The filenames starting with part* comes from acoustic woodwind recording, the ones starting with RWC are synthesized. The legend about the instruments are:

bs = bassoon

cl = clarinet

fl = flute

hn = horn

ob = oboe

vl = violin

cel = cello

gtr = guitar

sax saxophone

bass = electric bass guitar

Run Times

file /nema-raid/www/mirex/results/multiF0_task1_runtimes.csv not found

Overall Summary Results Task II

This subtask is evaluated in two different ways. In the first setup , a returned note is assumed correct if its onset is within +-50ms of a ref note and its F0 is within +- quarter tone of the corresponding reference note, ignoring the returned offset values. In the second setup, on top of the above requirements, a correct returned note is required to have an offset value within 20% of the ref notes duration around the ref note`s offset, or within 50ms whichever is larger. The Overlap ratio is calculated for an individual correctly identified note as File:2007 Overlap.PNG

A total of 30 files were used in this task: 16 real recordings, 8 synthesized from RWC samples, and 6 piano. The results below are the average of these 30 files.

Results Based on Onset Only

file /nema-raid/www/mirex/results/multiF0.note.onset.only.eval.csv not found

Results Based on Onset-Offset

file /nema-raid/www/mirex/results/multiF0.note.eval.csv not found

Piano Results based on Onset Only

file /nema-raid/www/mirex/results/multiF0.note.onset.only.eval.for.piano.csv not found

Individual Results Files for Task 2

For Onset only Evaluation

AC3 = Arshia Cont
AC4 = Arshia Cont
EV3 = Emmanuel Vincent, Nancy Bertin, Roland Badeau
EV4 = Emmanuel Vincent, Nancy Bertin, Roland Badeau
KE3 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama
KE4 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama
PE2 = Graham Poliner, Daniel P. W. Ellis
PI2 = Antonio Pertusa, José Manuel Iñesta 1
PI3 = Antonio Pertusa, José Manuel Iñesta 1
RK = Matt Ryynänen, Anssi Klapuri
VE2 = Valentin Emiya, Roland Badeau, Bertrand David

For Onset/Offset Evaluation

AC3 = Arshia Cont
AC4 = Arshia Cont
EV3 = Emmanuel Vincent, Nancy Bertin, Roland Badeau
EV4 = Emmanuel Vincent, Nancy Bertin, Roland Badeau
KE3 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama
KE4 = Koji Egashira, Hirokazu Kameoka, Shigeki Sagayama
PE2 = Graham Poliner, Daniel P. W. Ellis
PI2 = Antonio Pertusa, José Manuel Iñesta 1
PI3 = Antonio Pertusa, José Manuel Iñesta 1
RK = Matt Ryynänen, Anssi Klapuri
VE2 = Valentin Emiya, Roland Badeau, Bertrand David

Info About Filenames

The filenames starting with part* comes from acoustic woodwind recording, the ones starting with RWC are synthesized. The piano files are: RA_C030_align.wav,bach_847TESTp.wav,beet_pathetique_3TESTp.wav,mz_333_1TESTp.wav,scn_4TESTp.wav.note, ty_januarTESTp.wav.note

Run Times

file /nema-raid/www/mirex/results/multiF0task2.runtimes.csv not found