Difference between revisions of "2010:Runtime"

From MIREX Wiki
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
'''Info about run times''':
 +
There are various kinds of overheads involved in the running of people's algorithm including launching Matlab, remoting the streams and overhead of the workflow infrastructure.
 +
 
'''Task 4''' (MIREX 2010: Audio Train/Test: Genre Classification (Latin) - MIREX09 Dataset)
 
'''Task 4''' (MIREX 2010: Audio Train/Test: Genre Classification (Latin) - MIREX09 Dataset)
 
:Music classification task requiring participants to classify music audio tracks by Genre, where the genres are drawn from a set of Latin music genres collected in Brazil and annotated by experts.
 
:Music classification task requiring participants to classify music audio tracks by Genre, where the genres are drawn from a set of Latin music genres collected in Brazil and annotated by experts.

Latest revision as of 18:14, 26 October 2010

Info about run times: There are various kinds of overheads involved in the running of people's algorithm including launching Matlab, remoting the streams and overhead of the workflow infrastructure.

Task 4 (MIREX 2010: Audio Train/Test: Genre Classification (Latin) - MIREX09 Dataset)

Music classification task requiring participants to classify music audio tracks by Genre, where the genres are drawn from a set of Latin music genres collected in Brazil and annotated by experts.
Flow 39 -Mirex2010 Train and Classify template (one call format)
Submission Code Runtime
TN1 08:08:13

download these results as csv

Task 4 (MIREX 2010: Audio Train/Test: Genre Classification (Latin) - MIREX09 Dataset)

Music classification task requiring participants to classify music audio tracks by Genre, where the genres are drawn from a set of Latin music genres collected in Brazil and annotated by experts.
Flow 35 -Mirex2010 Feature Extraction
Submission Code Runtime
BMPE2 07:06:26
BRPC1 68:57:30
GP1 38:38:16
HE1 00:55:28
MBP1 03:06:47
MP2 16:04:11
RRS1 00:00:32
SL1 39:46:28
SSPK1 35:27:45
TN2 05:27:54
WLB1 11:18:53
WLJW2 00:00:48

download these results as csv

Task 4 (MIREX 2010: Audio Train/Test: Genre Classification (Latin) - MIREX09 Dataset)

Music classification task requiring participants to classify music audio tracks by Genre, where the genres are drawn from a set of Latin music genres collected in Brazil and annotated by experts.
Flow 36 -Mirex2010 Train and Classify task template - with precomputed features
Submission Code Runtime
BRPC1 00:46:12
GP1 00:00:45
HE1 10:31:54
MBP1 00:03:28
MP2 20:09:25
TN2 00:00:52
WLB1 00:45:20
WLB2 00:01:45
WLB3 00:15:21

download these results as csv

Task 5 (MIREX 2010: Audio Train/Test: Classical Composer Identification - MIREX08 Dataset)

Music classification task requiring participants to classify music audio tracks according by composer, where the composers and dataset are drawn from a set of western classical art music genres.
Flow 39 -Mirex2010 Train and Classify template (one call format)
Submission Code Runtime
TN1 01:11:56
TN4 01:03:02

download these results as csv

Task 5 (MIREX 2010: Audio Train/Test: Classical Composer Identification - MIREX08 Dataset)

Music classification task requiring participants to classify music audio tracks according by composer, where the composers and dataset are drawn from a set of western classical art music genres.
Flow 35 -Mirex2010 Feature Extraction
Submission Code Runtime
BMPE2 00:33:35
BRPC1 02:44:43
BRPC2 06:20:17
GP1 03:39:15
HE1 00:16:47
MBP1 00:17:44
MP2 00:23:27
RRS1 11:32:43
SSPK1 03:51:08
TN2 03:58:16
WLB1 19:41:01

download these results as csv

Task 5 (MIREX 2010: Audio Train/Test: Classical Composer Identification - MIREX08 Dataset)

Music classification task requiring participants to classify music audio tracks according by composer, where the composers and dataset are drawn from a set of western classical art music genres.
Flow 36 -Mirex2010 Train and Classify task template - with precomputed features
Submission Code Runtime
BMPE2 02:07:12
GP1 00:06:43
HE1 22:31:45
MBP1 00:03:19
TN2 01:09:11
WLB1 00:38:19
WLB2 00:01:23
WLB3 00:21:05
WLB4 00:08:41

download these results as csv

Task 8 (MIREX 2010: Audio Train/Test: Mood Classification - MIREX08 Dataset)

Music classification task requiring participants to classify music audio tracks by Mood, where the moods are drawn from a set of commercial production music.
Flow 35 -Mirex2010 Feature Extraction
Submission Code Runtime
BMPE2 00:04:32
BRPC1 01:54:16
BRPC2 00:36:56
GP1 02:00:01
HE1 00:09:30
MBP1 00:12:15
MP2 00:20:03
RRS1 00:05:01
SL1 14:49:42
SSPK1 02:03:29
TN2 00:58:40
TS1 00:00:58
WLB1 02:39:32
WLJW1 09:44:57
WLJW2 08:12:07

download these results as csv

Task 8 (MIREX 2010: Audio Train/Test: Mood Classification - MIREX08 Dataset)

Music classification task requiring participants to classify music audio tracks by Mood, where the moods are drawn from a set of commercial production music.
Flow 36 -Mirex2010 Train and Classify task template - with precomputed features
Submission Code Runtime
BMPE2 00:34:07
BRPC1 00:10:44
BRPC2 03:29:04
GP1 00:02:31
HE1 00:39:38
MBP1 00:01:08
MP2 07:53:30
RRS1 00:25:19
SSPK1 00:20:07
TN2 00:07:35
WLB1 00:10:48
WLB2 00:03:00
WLB3 00:04:29
WLB4 00:01:00

download these results as csv

Task 9 (MIREX 2010: Audio Melody Extraction - ADC04 Dataset)

Melody extraction from audio task requiring participants to annotate the dominant F0 at each 10ms timestep on the ADC04 dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
HJ1 00:30:53
JJY1 00:26:06
JJY2 00:36:17
SG1 01:05:58
TOOS1 00:04:26

download these results as csv

Task 11 (MIREX 2010: Audio Melody Extraction - MIREX05 Dataset)

Melody extraction from audio task requiring participants to annotate the dominant F0 at each 10ms timestep on the MIREX05 dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
HJ1 00:59:31
JJY1 00:50:31
JJY2 01:09:30
SG1 03:48:02
TOOS1 00:08:15

download these results as csv

Task 13 (MIREX 2010: Audio Melody Extraction - INDIAN08 Dataset)

Melody extraction from audio task requiring participants to annotate the dominant F0 at each 10ms timestep on the INDIAN08 dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
HJ1 00:38:18
JJY1 00:53:50
JJY2 00:52:14
SG1 11:27:29
TOOS1 00:05:53

download these results as csv

Task 14 (MIREX 2010: Audio Melody Extraction - MIREX09 0dB Dataset)

Melody extraction from audio task requiring participants to annotate the dominant F0 at each 10ms timestep on the MIREX09 0dB vocal to accompaniment ratio dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
HJ1 14:39:16
JJY1 12:07:21
JJY2 14:06:20
SG1 65:21:11
TOOS1 01:56:27

download these results as csv

Task 15 (MIREX 2010: Audio Melody Extraction - MIREX09 -5dB Dataset)

Melody extraction from audio task requiring participants to annotate the dominant F0 at each 10ms timestep on the MIREX09 -5dB vocal to accompaniment ratio dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
AR1 14:33:21
HJ1 10:58:35
JJY1 08:43:59
JJY2 14:10:22
SG1 48:21:52
TOOS1 01:31:01

download these results as csv

Task 17 (MIREX 2010: Audio Chord Description - MIREX09 Dataset)

Chord transcription task requiring participants to annotate and segment the chord events in the MIREX09chord transcription dataset.
Please note that:
  • Evaluations are performed at the triad level,
  • results for both pretrained algorithms and algorithms trained and tested under 3 fold cross-validation are reported here.
  • pretrained algorithms are likely to have been trained on the evaluation dataset hence they are expected to achieve higher results than algorithms evaluated on held out data.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
CWB1 02:53:50
EW3 01:12:41
EW4 00:45:25
JJY2 04:49:09
KO1 10:50:30
MD1 05:44:47
MM1 15:05:16
OFG1 00:27:22
PVM1 01:06:26
RRH1 07:29:00
RRHS1 08:41:56
RRHS2 07:23:41
RVCCR2 05:53:03
UUOS1 01:56:56

download these results as csv

Task 16 (MIREX 2010: Audio Melody Extraction - MIREX09 +5dB Dataset)

Melody extraction from audio task requiring participants to annotate the dominant F0 at each 10ms timestep on the MIREX09 +5dB vocal to accompaniment ratio dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
HJ1 11:16:37
JJY1 08:56:31
JJY2 13:55:51
SG1 59:31:31
TOOS1 01:56:56

download these results as csv

Task 19 (MIREX 2010: Audio Structural Segmentation - MIREX09 Dataset)

Structural Segmentation task requiring participants to analyse tracks and determine a (verse/chorus/bridge etc) structure and repeated sections in the music.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
BV1 21:43:13
BV2 24:26:04
MHRAF2 13:01:15
MND1 10:27:03
WB1 07:58:04

download these results as csv

Task 18 (MIREX 2010: Audio Tempo Extraction - MIREX06 Dataset)

Tempo estimation task requiring participants to estimate two tempo and relative strength between them for tracks from the MIREX 2006 Tempo estimation dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
BES2 02:22:14
ES1 00:02:19
GKC1 03:12:28
GT1 00:02:39
NW2 00:02:03
OL1 00:41:38
TL1 00:19:21

download these results as csv

Task 25 (MIREX 2010: Audio Onset Detection - MIREX05 Dataset)

Onset detection task requiring participants to analyse tracks and determine the onset times of audio events in the MIREX05 onset detection dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
AR2 00:03:53
AR3 00:04:02
AR4 00:03:52
AR5 00:03:46
BES1 00:00:20
BT1 00:54:05
BT2 00:53:46
BT3 00:53:06
BT4 00:58:26
ME1 00:06:56
RVCCR1 02:19:17
RVCCR2 00:44:33
TGGL1 00:01:52
TZC1 02:30:29
TZC2 03:39:24
TZC3 04:22:07
TZC4 03:49:30
TZC5 04:09:49

download these results as csv

Task 27 (MIREX 2010: Audio Key Detection - MIREX05 Dataset)

Music Key classification task requiring participants to classify music audio tracks (synthesized from MIDI files) according to Key.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
GP8 00:25:00
PVM2 07:02:07
RRH1 14:44:08
RRH2 11:01:22
UUOS2 01:37:51

download these results as csv

Task 26 (Demo: Classification Latin Genre)

Music classification task requiring participants to classify music audio tracks by Genre, where the genres are drawn from a set of Latin music genres collected in Brazil and annotated by experts. This is reduced size dataset for testing classifiers and demonstrations.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
BRPC2 00:00:48

download these results as csv

Task 26 (Demo: Classification Latin Genre)

Music classification task requiring participants to classify music audio tracks by Genre, where the genres are drawn from a set of Latin music genres collected in Brazil and annotated by experts. This is reduced size dataset for testing classifiers and demonstrations.
Flow 5 -Mirex2010 Test Train task template NEW
Submission Code Runtime
BRPC2 00:00:48

download these results as csv

Task 31 (Demo: Key Classification MIREX05)

Music Key classification task requiring participants to classify music audio tracks (synthesized from MIDI files) according to Key. A reduced size dataset is used to simplify testing and demonstration.
Flow 4 -Mirex2010 Feature Extraction
Submission Code Runtime
BRPC2 00:00:50

download these results as csv

Task 31 (Demo: Key Classification MIREX05)

Music Key classification task requiring participants to classify music audio tracks (synthesized from MIDI files) according to Key. A reduced size dataset is used to simplify testing and demonstration.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
BRPC2 00:00:48

download these results as csv

Task 31 (Demo: Key Classification MIREX05)

Music Key classification task requiring participants to classify music audio tracks (synthesized from MIDI files) according to Key. A reduced size dataset is used to simplify testing and demonstration.
Flow 5 -Mirex2010 Test Train task template NEW
Submission Code Runtime
BRPC2 00:00:25

download these results as csv

Task 34 (MIREX 2010: Audio Beat Tracking - MIREX09 MAZ Dataset)

Beat detection task requiring participants to analyse tracks and determine the beat times for tracks in the MIREX09 Beat detection dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
BES3 29:32:28
BES4 34:09:38
LGG1 00:57:34
LGG2 00:48:56
MRVCC2 53:49:15
NW1 00:23:14
TL2 02:01:39
ZTC1 04:17:39

download these results as csv

Task 33 (MIREX 2010: Audio Beat Tracking - MIREX06 MCK Dataset)

Beat detection task requiring participants to analyse tracks and determine the beat times for tracks in the MIREX06 Beat detection dataset.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
BES3 02:25:32
BES4 02:16:01
LGG1 00:09:44
LGG2 00:08:14
MRVCC1 14:01:43
NW1 00:05:16
TL2 00:24:00

download these results as csv

Task 37 (MIREX 2010: Audio Structural Segmentation - MIREX10 Dataset)

Structural Segmentation task requiring participants to analyse tracks and determine a (verse/chorus/bridge etc) structure and repeated sections in the music.
Flow 1 -Mirex2010 Analysis task template
Submission Code Runtime
BV1 15:41:16
BV2 09:28:00
MHRAF2 11:01:24
MND1 11:04:28
WB1 05:39:55

download these results as csv