2005:Call For Evaluation Topics

Deadline: January 16, 2005

As a first step, we are issuing a call for evaluation topics. Calls for evaluation procedures and for participation will follow.

Submissions of evaluation topics should include:

• a title
• a short description of the problem to be addressed (data and query type, expected results),
• a list of possible participants (names, email addresses, and likelihood of entering),
• suggestions for evaluation procedures (metrics, listening tests, etc),
• suggestions for relevant test collections (including how they might be obtained and possible copyright issues).

Proposals should be submitted to J. Stephen Downie (MIREX General Chair) and Emmanuel Vincent (MIREX Local Chair) by January 16, 2005.

The MIREX organizing committee will retain all proposals, score them according to their likelihood of being held and publish them on the MIREX mailing list and web site. All willing participants will have until February 27, 2005 to amend the proposals. Then the MIREX organizing committee will review the amended proposals and select the ones that will be held.