Difference between revisions of "2014:Singing Voice Separation"

From MIREX Wiki
m (Submission format)
m (Packaging submissions)
Line 66: Line 66:
  
 
== Packaging submissions ==
 
== Packaging submissions ==
 
 
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guarenteed).
 
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guarenteed).
 +
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]]
 +
# Be sure to follow the [[MIREX 2014 Submission Instructions]]
  
 
All submissions should include a README file including the following the information:
 
All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands  
+
# Command line calling format for all executables and an example formatted set of commands
 
# Number of threads/cores used or whether this should be specified on the command line
 
# Number of threads/cores used or whether this should be specified on the command line
 
# Expected memory footprint
 
# Expected memory footprint
 
# Expected runtime
 
# Expected runtime
# Any required environments (and versions), e.g. python, java, bash, matlab.
+
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
 +
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
 +
# Any special notice regarding to running your algorithm
 +
 
 +
Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.
  
 
== Time and hardware limits ==
 
== Time and hardware limits ==

Revision as of 23:04, 11 September 2014

Description

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

Task specific mailing list

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

Data

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms.

Collection statistics:

  1. Size of collection: 100 clips
  2. Audio details: 16-bit, mono, 44.1kHz, WAV
  3. Duration of each clip: 30 seconds

Evaluation

For evaluation we use Vincent et al.'s (2012) Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by bss_eval_sources.m in BSS Eval Version 3.0. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]', [trueVoice trueKaraoke]')

 SDR =
    -2.7443
     2.7514

 SIR =
    -0.5416
     9.0144

 SAR =
     4.5486
     4.4363

The final scores will be determined by the average scores over all 100 clips:

,

,

.

Submission format

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

Following the convention of other MIREX tasks, a two-to-four-page abstract is also required (see Packaging submissions below).

Packaging submissions

All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guarenteed).

  1. Be sure to follow the Best Coding Practices for MIREX
  2. Be sure to follow the MIREX 2014 Submission Instructions

All submissions should include a README file including the following the information:

  1. Command line calling format for all executables and an example formatted set of commands
  2. Number of threads/cores used or whether this should be specified on the command line
  3. Expected memory footprint
  4. Expected runtime
  5. Approximately how much scratch disk space will the submission need to store any feature/cache files?
  6. Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
  7. Any special notice regarding to running your algorithm

Note that the information that you place in the README file is extremely important in ensuring that your submission is evaluated properly.

Time and hardware limits

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

Potential Participants

name / email