2008:Audio Chord Detection

Introduction

For many applications in music information retrieval, extracting the harmonic structure is very desirable, for example for segmenting pieces into characteristic segments, for finding similar pieces, or for semantic analysis of music.

The extraction of the harmonic structure requires the detection of as many chords as possible in a piece. That includes the characterisation of chords with a key and type as well as a chronological order with onset and duration of the chords.

Although some publications are available on this topic [1,2,3,4,5], comparison of the results is difficult, because different measures are used to assess the performance. To overcome this problem an accurately defined methodology is needed. This includes a repertory of the findable chords, a defined test set along with ground truth and unambiguous calculation rules to measure the performance.

Regarding this we suggest to introduced the new evaluation task Audio Chord Detection.

Data

As this is intended for music information retrieval, the analysis should be performed on real world audio, not resynthesized MIDI or special renditions of single chords. We suggest the test bed consists of WAV-files in CD quality (with a sampling rate of 44,1kHz and a solution of 16 bit). A representative test bed should consist of more than 50 songs of different genres like pop, rock, jazz and so on.

For each song in the test bed, a ground truth is needed. This should comprise all detectable chords in this piece with their tonic, type and temporal position (onset and duration) in a machine readable format that is still to be specified.

To define the ground truth, a set of detectable chords has to be identified. We propose to use the following set of chords build upon each of the twelve semitones.

Triads: major, minor, diminished, augmented, suspended4
 Quads: major-major 7, major-minor 7, major add9, major maj7/#5 
        minor-major 7, minor-minor 7, minor add9, minor 7/b5
        maj7/sus4, 7/sus4

An approach for text annotation of musical chords is presented in [6].

We could contribute excerpts of approximately 30 pop and rock songs including a ground truth.

Evaluation

Two common measures from field of information retrieval are recall and precision. They can be used to evaluate a chord detection system.

Recall: number of time units where the chords have been correctly identified by the algorithm divided by the number of time units which contain detectable chords in the ground truth.

Precision: number of time units where the chords have been correctly identified by the algorithm divided by the total number of time units where the algorithm detected a chord event.

Points to discuss:

- Are the measures mentioned above sufficient to evaluate the algorithms? In particular: Can an algorithm which achieves high precision and recall on many time units, but has an otherwise "jagged" output (i.e. is wrong often, but for a short time) be considered as good as a smoother one with equal precision and recall?

- Should chord data be expressed in absolute (aka "F major-minor 7") or relative (aka "C: IV major-minor 7") terms?

- Should different inversions of chords be considered in the evaluation process?

- What temporal resolution should be used for ground truth and results?

- How should enharmonic and other confusions of chords be handled?

- How will Ground Truth be determined?

- What degree of chordal/tonal complexity will the music contain?

- Will we include any atonal or polytonal music in the Ground Truth dataset?

- What is the maximal acceptable onset deviation between ground truth and result?

- What file format should be used for ground truth and output?

Potential Participants

• H.Papadopoulos (papadopo@ircam.fr)
• Jan Weil (weil@nue.tu-berlin.de)
• Markus Mehnert, Gabriel Gatzsche (markus.mehnert@tu-ilmenau.de, gze@idmt.fraunhofer.de)
• Yuki Uchiyama(uchiyama@hil.t.u-tokyo.ac.jp)
• Matti Ryyn├ñnen and Anssi Klapuri (Tampere University of Technology), matti.ryynanen <at> tut.fi, anssi.klapuri <at> tut.fi
• Xinglin Zhang (University of Regina, zhang46x@uregina.ca)

Bibliography

1.Harte,C.A. and Sandler,M.B.(2005). Automatic chord identification using a quantised chromagram. Proceedings of 118th Audio Engineering Society's Convention.

2.Sailer,C. and Rosenbauer K.(2006). A bottom-up approach to chord detection. Proceedings of International Computer Music Conference 2006.

3.Shenoy,A. and Wang,Y.(2005). Key, chord, and rythm tracking of popular music recordings. Computer Music Journal 29(3), 75-86.

4.Sheh,A. and Ellis,D.P.W.(2003). Chord segmentation and recognition using em-trained hidden markov models. Proceedings of 4th International Conference on Music Information Retrieval.

5.Yoshioka,T. et al.(2004). Automatic Chord Transcription with concurrent recognition of chord symbols and boundaries. Proceedings of 5th International Conference on Music Information Retrieval.

6.Harte,C. and Sandler,M. and Abdallah,S. and G├│mez,E.(2005). Symbolic representation of musical chords: a proposed syntax for text annotations. Proceedings of 6th International Conference on Music Information Retrieval.

7.Papadopoulos,H. and Peeters,G.(2007). Large-scale study of chord estimation algorithms based on chroma representation and HMM. Proceedings of 5th International Conference on Content-Based Multimedia Indexing.

Comments/Discussion

Matti's comments (06/08/08).

Hi all, I'm glad to see several potential participants to this new task. Should we start to decide the output format for the submissions?

Although it would be nice to have a large set of possible chords in the evaluation, I suggest performing the evaluation based on triads alone, even perhaps only on major and minor thirds if just the evaluation data is suitable for this. In this case, the output format could be <white-space><chord identifier>, where the time stamp would be in seconds and the chord identifier an integer between 0-23, where values 0-11 denote C major, C# major, D major, and so forth until B major, and similarly values between 12-23 C minor, C# minor etc. The time stamps could denote either the chord onset times or the frame times (e.g., on a 10-ms grid). Should the evaluation take silent or no-chord segments into account? Any comments or ideas are welcome.

Kyogu's comments (August 7, 2008)

I'm with Matti in reducing the types of chords only to triads or even to major/minor triads. Since this is going to be the first chord detection task (if it ever happens), we can start from a simple task and later move on to more difficult tasks after analyzing the results. As to the output format, we'd better stick to absolute times in (milli)seconds than to the frame numbers since they should vary from algo to algo.

Xinglin Zhang's comments(August 7, 2008)

Matti's suggest sounds good. But don't forget that some parts of the song do not have a chord (for example,at the beginning of the song where there is nothing or there are only drums going on). Thus 0-23 is not enough. We can introduce a 24 for none-chord or we can use the representations give by Reference 5 on this page, if you have read it. Let's determine the format as soon as possible. BTW, I sent Kyogu an email a couple of weeks ago on some problems regarding chord recognition, but I received no response. I am not sure whether you have got that email or not. Thanks.