Difference between revisions of "2025:Audio Beat Tracking Results"

From MIREX Wiki
(→‎Submissions)
(→‎SMC)
 
(9 intermediate revisions by 2 users not shown)
Line 23: Line 23:
 
| KG-ApolloBeats
 
| KG-ApolloBeats
 
| The 2025 KG Music Beats Tracking System
 
| The 2025 KG Music Beats Tracking System
 +
| TBA
 +
| DingKun Xiao, Haijun Cai, Chuanyi Chen
 +
|-
 +
| KG2
 +
| Same as KG-ApolloBeats, but not trained on SMC or GTZAN
 
| TBA
 
| TBA
 
| DingKun Xiao, Haijun Cai, Chuanyi Chen
 
| DingKun Xiao, Haijun Cai, Chuanyi Chen
 
|-  
 
|-  
 
| BeatU
 
| BeatU
−
| BeatU
+
| Beat-U: Multi-Task Music Understanding with Hierarchical Timescales
 
| TBA
 
| TBA
−
| Jingwei Zhao
+
| YAMAHA Corporation
 
|}
 
|}
  
Line 36: Line 41:
 
* '''GTZAN''': 999 songs from the GTZAN dataset (starting from next year, training on GTZAN will be disallowed)
 
* '''GTZAN''': 999 songs from the GTZAN dataset (starting from next year, training on GTZAN will be disallowed)
 
* '''SMC''': 217 songs from the SMC collection
 
* '''SMC''': 217 songs from the SMC collection
−
* '''Yamaha_JPOP''': A private dataset annotated by Yamaha Corporation. The dataset contains 200 JPOP songs.
+
* '''Yamaha_JPOP''': A private dataset annotated by Yamaha Corporation. The dataset contains 250 JPOP songs.
−
* '''Yamaha_Balanced''': A private dataset annotated by Yamaha Corporation. The dataset contains 241 songs. While it is still biased towards JPOP songs, the dataset covers a wider range of genres: J.Pop (10.37%), Rock (10.37%), J.Enka (10.37%), J.Kayoukyoku (10.37%), Soundtrack (10.37%), Western Pop (10.37%), Children's Song (10.37%), R&B (6.22%), Hiphop (4.56%), Jazz (2.49%), Dance (2.49%), World (2.07%), Techno (1.24%), Easy listening (1.24%), J.Minyou (1.24%), Others (5.81%).
+
* '''Yamaha_Balanced''': A private dataset annotated by Yamaha Corporation. The dataset contains 239 songs. While it is still biased towards JPOP songs, the dataset covers a wider range of genres: J.Pop (10.37%), Rock (10.37%), J.Enka (10.37%), J.Kayoukyoku (10.37%), Soundtrack (10.37%), Western Pop (10.37%), Children's Song (10.37%), R&B (6.22%), Hiphop (4.56%), Jazz (2.49%), Dance (2.49%), World (2.07%), Techno (1.24%), Easy listening (1.24%), J.Minyou (1.24%), Others (5.81%).
−
 
 
 
== GTZAN ==
 
== GTZAN ==
  
Line 72: Line 76:
 
| 86.86
 
| 86.86
 
| 92.77
 
| 92.77
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | KG2
 +
| 88.21
 +
| 76.45
 +
| 75.28
 +
| 88.38
 +
| 78.48
 +
| 81.05
 +
| 88.63
 +
| 92.18
 
|- style="vertical-align:bottom;"
 
|- style="vertical-align:bottom;"
 
| style="text-align:left;" | Baseline: BeatThis
 
| style="text-align:left;" | Baseline: BeatThis
Line 93: Line 107:
 
| 86.69
 
| 86.69
 
|}
 
|}
 +
  
 
Entries with [*] are trained on this dataset.
 
Entries with [*] are trained on this dataset.
Line 131: Line 146:
 
| 55.96
 
| 55.96
 
| 76.22
 
| 76.22
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | KG2
 +
| 58.55
 +
| 45.08
 +
| 25.81
 +
| 68.36
 +
| 39.14
 +
| 50.80
 +
| 46.81
 +
| 61.45
 
|- style="vertical-align:bottom;"
 
|- style="vertical-align:bottom;"
 
| style="text-align:left;" | Baseline: BeatThis*
 
| style="text-align:left;" | Baseline: BeatThis*
Line 152: Line 177:
 
| 29.48
 
| 29.48
 
|}
 
|}
−
 
 
Entries with [*] are trained on this dataset.
 
Entries with [*] are trained on this dataset.
  
Line 264: Line 288:
 
| 85.29
 
| 85.29
 
|}
 
|}
 +
 +
== Comparison with Previous MIREXes ==
 +
 +
Since this year we have switched to using mir_eval for evaluation, some results may differ from those in previous MIREX editions due to differences in implementation. We confirm that the following metrics remain comparable with previous MIREX results:
 +
 +
* Comparable: F1, Goto, CMLc, CMLt, AMLc, AMLt.
 +
* Not comparable: Cemgil, P-score.

Latest revision as of 00:09, 14 September 2025

Submissions

This page is still WIP. More submissions might appear later.

Submission Title PDF Authors
Baseline: CD1 QM Tempo Tracker Link
Baseline: BeatThis Beat This! Accurate Beat Tracking Without DBN Postprocessing Link
KG-ApolloBeats The 2025 KG Music Beats Tracking System TBA DingKun Xiao, Haijun Cai, Chuanyi Chen
KG2 Same as KG-ApolloBeats, but not trained on SMC or GTZAN TBA DingKun Xiao, Haijun Cai, Chuanyi Chen
BeatU Beat-U: Multi-Task Music Understanding with Hierarchical Timescales TBA YAMAHA Corporation

Test Sets

  • GTZAN: 999 songs from the GTZAN dataset (starting from next year, training on GTZAN will be disallowed)
  • SMC: 217 songs from the SMC collection
  • Yamaha_JPOP: A private dataset annotated by Yamaha Corporation. The dataset contains 250 JPOP songs.
  • Yamaha_Balanced: A private dataset annotated by Yamaha Corporation. The dataset contains 239 songs. While it is still biased towards JPOP songs, the dataset covers a wider range of genres: J.Pop (10.37%), Rock (10.37%), J.Enka (10.37%), J.Kayoukyoku (10.37%), Soundtrack (10.37%), Western Pop (10.37%), Children's Song (10.37%), R&B (6.22%), Hiphop (4.56%), Jazz (2.49%), Dance (2.49%), World (2.07%), Techno (1.24%), Easy listening (1.24%), J.Minyou (1.24%), Others (5.81%).

GTZAN

Group F1 Cemgil Goto P-score CMLc CMLt AMLc AMLt
BeatU 84.93 68.23 64.06 84.27 71.07 75.26 78.78 84.27
KG-ApolloBeats* 92.53 80.66 79.38 93.55 83.44 88.49 86.86 92.77
KG2 88.21 76.45 75.28 88.38 78.48 81.05 88.63 92.18
Baseline: BeatThis 89.02 80.15 72.27 88.00 76.00 79.64 84.63 90.01
Baseline: CD1 81.19 69.64 62.06 79.97 65.02 66.94 83.08 86.69


Entries with [*] are trained on this dataset.

The baseline BeatThis reports different results compared to the paper because it uses a different number of test songs (999 vs. 993).

SMC

Group F1 Cemgil Goto P-score CMLc CMLt AMLc AMLt
BeatU 53.14 40.67 14.75 63.52 27.24 41.16 30.88 47.44
KG-ApolloBeats* 74.15 57.07 32.72 84.51 53.73 72.66 55.96 76.22
KG2 58.55 45.08 25.81 68.36 39.14 50.80 46.81 61.45
Baseline: BeatThis* 71.81 55.64 27.19 82.91 49.78 69.89 51.15 72.30
Baseline: CD1 33.66 26.29 6.91 45.10 9.88 13.12 17.99 29.48

Entries with [*] are trained on this dataset.

YAMAHA_Balanced

Group F1 Cemgil Goto P-score CMLc CMLt AMLc AMLt
BeatU 92.55 82.28 89.12 93.57 83.62 90.15 85.84 93.11
KG-ApolloBeats 91.97 81.29 88.28 92.54 79.91 88.30 82.26 91.79
Baseline: BeatThis 90.59 79.43 81.59 91.42 64.94 84.52 66.87 87.73
Baseline: CD1 76.43 67.85 64.44 74.42 55.13 59.47 71.86 83.63

YAMAHA_JPop

Group F1 Cemgil Goto P-score CMLc CMLt AMLc AMLt
BeatU 96.58 88.94 95.20 96.73 92.32 94.46 94.65 97.05
KG-ApolloBeats 95.39 86.66 93.20 94.63 82.87 90.54 84.72 93.48
Baseline: BeatThis 94.00 84.08 86.80 93.15 69.66 86.64 71.39 89.57
Baseline: CD1 77.38 70.76 64.40 73.55 54.98 58.71 77.28 85.29

Comparison with Previous MIREXes

Since this year we have switched to using mir_eval for evaluation, some results may differ from those in previous MIREX editions due to differences in implementation. We confirm that the following metrics remain comparable with previous MIREX results:

  • Comparable: F1, Goto, CMLc, CMLt, AMLc, AMLt.
  • Not comparable: Cemgil, P-score.
Retrieved from "https://www.music-ir.org/mirex/w/index.php?title=2025:Audio_Beat_Tracking_Results&oldid=14819"