Call For Test Data And Evaluation Procedures

From MIREX2005

Deadline: February 27, 2005

After the first MIREX call, ten detailed tasks were proposed for evaluation:

• audio artist identification,
• audio drum detection,
• audio genre classification,
• audio key finding,
• audio melody extraction,
• audio onset detection,
• audio tempo extraction,
• symbolic genre classification,
• symbolic key finding,
• symbolic melodic similarity.

These proposals have been reviewed regarding both scientific quality and technical feasibility and made available along with the reviews on a MIREX Wiki. Interested individuals should head right now to the Wiki and start making comments/suggestions on the proposals.

We are now issuing a second call for the willing participants to revise the proposals by February 27, 2005. More precisely:

• the text of the proposals should be modified to take the reviews into account,
• each proposer should contact the possible participants he mentioned to know if they will actually submit algorithms,
• participants should agree on evaluation procedures,
• participants should provide a detailed list (number of files, duration, etc) of the annotated test data that they already own and send these data to the organizing committee,
• participants should contact the organizing committee to discuss how much new copyrighted/uncopyrighted data could be gathered and annotated and how much time it would take.

Amended proposals should be very precise about the number of actual participants, the evaluation procedures and the amount of work needed to annotate new data needed. The MIREX organizing committee will review these proposals again and select the ones that will be held. Participants will then have about three months to submit their algorithms.

Discussion between participants may take place on the MIREX mailing list or on the Wiki, but revisions of the proposals should be performed on the Wiki only. The organizing committee contact for test data issues is J. Stephen Downie (MIREX General Chair, jdownie@uiuc.edu). Questions regarding other issues may be sent to Emmanuel Vincent (MIREX Local Chair, emmanuel.vincent@elec.qmul.ac.uk).

We are looking forward to your contributions!

- The MIREX organizing committee.