University of Illinois · Graduate School of Library and Information Science · ISRL

2005 MIREX Contest Results - Symbolic Melodic Similarity  (Contest wiki)


Goal: To retrieve the most similar incipits from the RISM A/II collection, given one of the incipits as a query.

Dataset: 558 MusicXML files of the incipits from the RISM collection. 11 incipits of this collection are chosen as queries. These MusicXML files were converted to MIDI files. 

The scores reported here are averaged over the 11 queries.

Rank Participant Average
Dynamic Recall 
Normalized  Recall at Group Boundaries

Average precision (non-interpolated)

Precision at N documents (N is number of relevant document)  Input Data Format Runtime (s) Machine
Detailed Evaluation Result
1 Grachten, Arcos & Mántaras
65.98% 55.24% 51.72% 44.33%
MIDI
80.174 * B0 GAM_eval.txt
2 Orio, N. 64.96% 53.35%
42.96%
39.86% XML 24.610 B4 O_eval.txt
3 Suyoto & Uitdenbogerd
64.18% 51.79%
40.42%
41.72% MIDI 48.133
B3
SU_eval.txt
4 Typke, Wiering & Veltkamp
57.09% 48.17%
35.64%
33.46% MIDI 51240 B4
TWV_eval.txt
5 Lemström, Mikkilä, Mäkinen & Ukkonen (P3) 55.82% 46.56% 41.40%
39.18% MIDI 10.007 * B0 LMMU_P3_eval.txt
6 Lemström, Mikkilä, Mäkinen & Ukkonen (DP)
54.27% 47.26%
39.91%
36.20% MIDI
10.106 * B0 LMMU_DP_eval.txt
7 Frieler &  Müllensiefen
51.81% 45.10%
33.93%
33.71% MIDI
54.593
B4 FM_eval.txt

Note * : These runs were executed in M2K environment, and thus the runtime includes evaluation time.



Maintained by :J Stephen Downie
Comments to : jdownie at uiuc dot edu
Last modified: 28 September 2005