2006:Main Page

MIREX 2006 RESULTS

The results are now available. The central listing is available on the 2006:MIREX2006 Results page. The abstracts are also available here.

NOTE: Due to an evaluation error, the results of the 2006:Audio Beat Tracking task have been updated as of 17 July 2007, and differ from those presented at ISMIR 2006.

Welcome to MIREX 2006

This is the main page for the 2006 running of the Music Information Retrieval Evaluation eXchange (MIREX 2006). The International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL) at the Graduate School of Library and Information Science (GSLIS), University of Illinois at Urbana-Champaign (UIUC) is hosting and running MIREX 2006.

We wish to thank the Task Leaders who have volunteered to co-ordinate the zillion details required to conduct the scientific evaluation of the wide variety of MIR-related tasks being explored by MIR researchers from around the globe. We also wish to thank the MIR community members for their constant input, commentary and suggestions on making the evaluation tasks possible. Special thanks to the Graduate Assistants at IMIRSEL who are working ever-so-diligently to make MIREX 2006 run even better than MIREX 2005.

Cheers,
J. Stephen Downie
Director, IMIRSEL

MIREX 2006 Submission Instructions

The 2006:MIREX 2006 Submission Instructions page is now up. The URL to the MIREX 2006 Submission System has been placed at the bottom of the MIREX 2006 Submission Instructions page (to encourage reading of this important document).

Andreas Ehmann has created another must-read submission-related document: 2006:Best Coding Practices for MIREX.

Take a look at our most recent description of MIREX 2006 Equipment and basic software.

If you have difficulties or general questions about the submission procedures please contact:

• mrx-com09@lists.lis.uiuc.edu

MIREX 2006 at ISMIR 2006

• MIREX 2006 Poster Session: Wednesday, 11 October 2006, during the lunch/poster session from 12:00-14:00.
• MIREX Results and Planning Plenary: The plenary results and planning meeting will be held Thursday, 12 October 2006 in the afternoon after lunch so we can be more flexible with timing and the meeting can stretch longer as necessary. I suspect that we could easily go for quite some time as we really need to discuss such important issues as organizational structures and the increasingly important "lack of training/testing/ground-truth" data issue.

MIREX 2006 Preliminary Timelines

• 24 July 2006: Submission System Opened: Code and First-Pass Extended Abstract Submissions
• 25 July 2006 through early September: Run evaluations (Task-specific deadlines TBA)
• 14 August 2006: Deadline for 2006:Audio Music Similarity and Retrieval and 2006:Symbolic Melodic Similarity submissions.
• 25 September 2006: Revised Abstracts Due (Tentative, depends on printing arrangements, could be earlier or later)

SPECIAL COMMENT ABOUT DEADLINES

This year, two tasks, 2006:Audio Music Similarity and Retrieval and 2006:Symbolic Melodic Similarity will require post-run human evaluations. Therfore we are going to press hard to have the running phases of these two tasks completed by 21 August 2006. It is also important to note that both these tasks MUST HAVE each entrant run before we can move on to the human evaluation phase (i.e., no stragglers). We will be able to extend the running of the other contests later into August and early September as these involve automatic evaluation procedures and no dependencies across systems.

All runs for 2006:Audio Music Similarity and Retrieval and 2006:Symbolic Melodic Similarity must be complete by August 21st in order to take part in the human evaluation. This means that the final submission deadline falls on August 14th at the absolute latest! No submission will be considered after this date (there will be absolutely no extensions to this deadline) and your submission must actually work by the 14th (no additional debugging time). Therefore, it is highly recommended that you submit before this date and ensure that you have carefully followed all specifications. Remember this is not a flexible deadline and you should submit long before it falls.

If you have problems with the 14 August 2006 deadline please contact the MIREX team immediately to see what, if any, accommodations can be made. The more lead time we at MIREX have, the higher the probability that some accommodations might be possible. Contact: mrx-com09@lists.lis.uiuc.edu .

MIREX 2006 Evaluation Tasks

As of 24 July 2006, these are the tasks that we plan to run as part of MIREX 2006

• 2006:Audio Beat Tracking
• 2006:Audio Melody Extraction
• 2006:Audio Music Similarity and Retrieval
• 2006:Audio Cover Song
• 2006:Audio Onset Detection
• 2006:Audio Tempo Extraction
• 2006:QBSH: Query-by-Singing/Humming
• Score Following
• 2006:Symbolic Melodic Similarity

Special Pairings of MIREX 2006 Tasks

This year, we seem to have arrived at some sets of evaluation tasks that are closely related to each other in some way (i.e., similar data, similar evaluations, and/or similar algorithms that can perform both tasks, etc.). Given these pairings, do be sure to check out the related tasks to see if you might want to participate. These pairings are below.

• 2006:Audio Beat Tracking and 2006:Audio Tempo Extraction
• 2006:Audio Music Similarity and Retrieval and 2006:Audio Cover Song
• 2006:Symbolic Melodic Similarity and 2006:QBSH: Query-by-Singing/Humming

MIREX 2005 and 2006 Wikis

This is the new wiki for MIREX 2006. The wiki for MIREX 2005 is still available at https://www.music-ir.org/mirex2005/. You can interlink between the 2005 and 2006 wikis using 2005: prefix on links to link to pages in MIREX2005.

Ideas for MIREX 2006 from MIREX 2005

Below are some ideas pilfered from the MIREX 2005 competition. As of 18 July 2006, we will NOT be running these tasks as part of MIREX 2006. We have established a 28 July 2006 DROP DEAD deadline for notifying us at IMIRSEL that at least 3 participants (representing different 3 different algorithms) are planning on participating in the running of one of these tasks.

• 2006:Audio Genre Classification
• 2006:Audio Drum Detection
• 2006:Melody Transcription from Monophonic Audio Recordings (it is NOT the same as melody extraction)
• 2006:Audio Singer Identification
• 2006:Audio and Symbolic Key Finding
• 2006:Symbolic Genre Classification

Task descriptions from MIREX 2005

• 2005:Audio Artist Identification
• 2005:Audio Drum Detection
• 2005:Audio Genre Classification
• 2005:Audio Melody Extraction
• 2005:Audio Onset Detection
• 2005:Audio Tempo Extraction
• 2005:Audio and Symbolic Key Finding
• 2005:Symbolic Genre Classification
• 2005:Symbolic Melodic Similarity

--Admin 13:51, 21 July 2006 (CDT)