Difference between revisions of "2007:Query-by-Singing/Humming Results"
From MIREX Wiki
CameronJones (talk | contribs) (→Task 2 Overall Results) |
(→Team ID) |
||
| (11 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
| − | These are the results for the 2007 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the [[Query by Singing/Humming]] page. | + | These are the results for the 2007 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the [[2007:Query by Singing/Humming]] page. |
===Task Descriptions=== | ===Task Descriptions=== | ||
| − | '''Task 1 [[#Task 1 Results|Goto Task 1 Results]]''': The first subtask is the same as last years. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. The query database consists of 2797 sung queries. | + | '''Task 1 [[#Task 1 Results|Goto Task 1 Results]]''': The first subtask is the same as last years. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. See [http://www.esac-data.org/ ESAC Data Homepage] for more information about the Essen Collection. The query database consists of 2797 sung queries. |
'''Task 2 [[#Task 2 Results|Goto Task 2 Results]]''': In the second subtask, the same setup as the first subtask used with combination of different transcribers and matchers. The test databases consists of 106 ground-truth MIDIS + 2000 Essen Collection MIDI noise files. The query databases consists of 355 sung queries. | '''Task 2 [[#Task 2 Results|Goto Task 2 Results]]''': In the second subtask, the same setup as the first subtask used with combination of different transcribers and matchers. The test databases consists of 106 ground-truth MIDIS + 2000 Essen Collection MIDI noise files. The query databases consists of 355 sung queries. | ||
| Line 10: | Line 10: | ||
===General Legend=== | ===General Legend=== | ||
====Team ID==== | ====Team ID==== | ||
| − | '''FH''' = [https://www.music-ir.org/ | + | '''FH''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_ferraro.pdf Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine]<br /> |
| − | '''CG''' = [https://www.music-ir.org/ | + | '''CG''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_SMS_gomez.pdf Carlos Gómez, Soraya Abad-Mota, Edna Ruckhaus]<br /> |
| − | '''RJ1''' = [https://www.music-ir.org/ | + | '''RJ1''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1]<br /> |
| − | '''RJ2''' = [https://www.music-ir.org/ | + | '''RJ2''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2]<br /> |
| − | '''NM''' = [https://www.music-ir.org/ | + | '''NM''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_lemstrom.pdf Kjell Lemström, Niko Mikkilä]<br /> |
| − | '''XW1''' = [https://www.music-ir.org/ | + | '''XW1''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_wu.pdf Xiao Wu, Ming Li 1]<br /> |
| − | '''XW2''' = [https://www.music-ir.org/ | + | '''XW2''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_wu.pdf Xiao Wu, Ming Li 2]<br /> |
| − | '''AU1''' = [https://www.music-ir.org/ | + | '''AU1''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_SMS_uitdenbogerd.pdf Alexandra L. Uitdenbogerd 1]<br /> |
| − | '''AU2''' = [https://www.music-ir.org/ | + | '''AU2''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_SMS_uitdenbogerd.pdf Alexandra L. Uitdenbogerd 2]<br /> |
| − | '''AU3''' = [https://www.music-ir.org/ | + | '''AU3''' = [https://www.music-ir.org/mirex/abstracts/2007/QBSH_SMS_uitdenbogerd.pdf Alexandra L. Uitdenbogerd 3]<br /> |
===Task 1 Results=== | ===Task 1 Results=== | ||
| Line 25: | Line 25: | ||
=====Task 1 Overall Results===== | =====Task 1 Overall Results===== | ||
| − | <csv> | + | <csv>2007/qbsh07_task1_overall.csv</csv> |
| − | |||
====Task 1 Friedman's Test for Significant Differences==== | ====Task 1 Friedman's Test for Significant Differences==== | ||
The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. | The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. | ||
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); | Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); | ||
| − | <csv>qbsh07_task1_sum_friedmans.csv</csv> | + | <csv>2007/qbsh07_task1_sum_friedmans.csv</csv> |
| − | <csv>qbsh07_task1_detail_friedmans.csv</csv> | + | <csv>2007/qbsh07_task1_detail_friedmans.csv</csv> |
| − | [[Image: | + | [[Image:2007_qbsh07_task1_friedmans.png]] |
====Task 1 Summary Results by Query Group==== | ====Task 1 Summary Results by Query Group==== | ||
| − | <csv>qbsh07_task1_avg_per_group.csv</csv> | + | <csv>2007/qbsh07_task1_avg_per_group.csv</csv> |
===Task 2 Results=== | ===Task 2 Results=== | ||
| Line 44: | Line 43: | ||
=====Team ID===== | =====Team ID===== | ||
'''FH_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_ferraro.pdf Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine based on XW note transcriber]<br /> | '''FH_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_ferraro.pdf Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine based on XW note transcriber]<br /> | ||
| − | '''CG_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_SMS_gomez.pdf Carlos | + | '''CG_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_SMS_gomez.pdf Carlos Gómez, Soraya Abad-Mota, Edna Ruckhaus based on XW note transcriber]<br /> |
'''RJ1_RJ''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on RJ pitch transcriber]<br /> | '''RJ1_RJ''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on RJ pitch transcriber]<br /> | ||
'''RJ1_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on XW pitch transcriber]<br /> | '''RJ1_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on XW pitch transcriber]<br /> | ||
'''RJ2_RJ''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on RJ pitch transcriber]<br /> | '''RJ2_RJ''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on RJ pitch transcriber]<br /> | ||
'''RJ2_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on XW pitch transcriber]<br /> | '''RJ2_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on XW pitch transcriber]<br /> | ||
| − | '''NM_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_lemstrom.pdf Kjell | + | '''NM_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_lemstrom.pdf Kjell Lemström, Niko Mikkilä based on XW note transcriber]<br /> |
'''XW1_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_wu.pdf Xiao Wu, Ming Li 1 based on XW note transcriber]<br /> | '''XW1_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_wu.pdf Xiao Wu, Ming Li 1 based on XW note transcriber]<br /> | ||
'''XW2_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_wu.pdf Xiao Wu, Ming Li 2 based on XW pitch transcriber]<br /> | '''XW2_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_wu.pdf Xiao Wu, Ming Li 2 based on XW pitch transcriber]<br /> | ||
=====Task 2 Overall Results===== | =====Task 2 Overall Results===== | ||
| − | <csv>qbsh07_task2_overall.csv</csv> | + | <csv>2007/qbsh07_task2_overall.csv</csv> |
====Task 2 Friedman's Test for Significant Differences==== | ====Task 2 Friedman's Test for Significant Differences==== | ||
The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. | The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. | ||
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); | Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); | ||
| − | <csv>qbsh07_task2_sum_friedmans.csv</csv> | + | <csv>2007/qbsh07_task2_sum_friedmans.csv</csv> |
| − | <csv>qbsh07_task2_detail_friedmans.csv</csv> | + | <csv>2007/qbsh07_task2_detail_friedmans.csv</csv> |
| − | [[Image: | + | [[Image:2007_qbsh07_task2_friedmans.png]] |
====Task 2 Summary Results by Query Group==== | ====Task 2 Summary Results by Query Group==== | ||
| − | <csv>qbsh07_task2_avg_per_group.csv</csv> | + | <csv>2007/qbsh07_task2_avg_per_group.csv</csv> |
[[Category: Results]] | [[Category: Results]] | ||
Latest revision as of 11:43, 26 July 2010
Contents
Introduction
These are the results for the 2007 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the 2007:Query by Singing/Humming page.
Task Descriptions
Task 1 Goto Task 1 Results: The first subtask is the same as last years. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The query database consists of 2797 sung queries.
Task 2 Goto Task 2 Results: In the second subtask, the same setup as the first subtask used with combination of different transcribers and matchers. The test databases consists of 106 ground-truth MIDIS + 2000 Essen Collection MIDI noise files. The query databases consists of 355 sung queries.
General Legend
Team ID
FH = Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine
CG = Carlos Gómez, Soraya Abad-Mota, Edna Ruckhaus
RJ1 = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1
RJ2 = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2
NM = Kjell Lemström, Niko Mikkilä
XW1 = Xiao Wu, Ming Li 1
XW2 = Xiao Wu, Ming Li 2
AU1 = Alexandra L. Uitdenbogerd 1
AU2 = Alexandra L. Uitdenbogerd 2
AU3 = Alexandra L. Uitdenbogerd 3
Task 1 Results
The first subtask is the same as last years. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. The query database consists of 2797 sung queries.
Task 1 Overall Results
| Teams | AU1 | AU2 | AU3 | CG | NM | FH | RJ1 | RJ2 | XW1 | XW2 |
|---|---|---|---|---|---|---|---|---|---|---|
| Task 1 (MRR) | 0.240 | 0.093 | 0.110 | 0.477 | 0.576 | 0.355 | 0.704 | 0.872 | 0.909 | 0.925 |
Task 1 Friedman's Test for Significant Differences
The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
| Friedman's ANOVA Table | |||||
|---|---|---|---|---|---|
| Source | SS | df | MS | Chi-sq | Prob>Chi-sq |
| Columns | 3.5829e+003 | 9 | 398.0984 | 393.0438 | 0 |
| Error | 355.1146 | 423 | 0.8395 | ||
| Total | 3938 | 479 |
| TeamID | TeamID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| AU1 | AU2 | -0.3039 | 1.6458 | 3.5956 | FALSE |
| AU1 | AU3 | -0.4706 | 1.4792 | 3.4289 | FALSE |
| AU1 | CG | -4.1164 | -2.1667 | -0.2169 | TRUE |
| AU1 | NM | -4.6998 | -2.75 | -0.8002 | TRUE |
| AU1 | FH | -2.6373 | -0.6875 | 1.2623 | FALSE |
| AU1 | RJ1 | -5.5435 | -3.5938 | -1.644 | TRUE |
| AU1 | RJ2 | -7.106 | -5.1563 | -3.2065 | TRUE |
| AU1 | XW1 | -7.5539 | -5.6042 | -3.6544 | TRUE |
| AU1 | XW2 | -8.2414 | -6.2917 | -4.3419 | TRUE |
| AU2 | AU3 | -2.1164 | -0.1667 | 1.7831 | FALSE |
| AU2 | CG | -5.7623 | -3.8125 | -1.8627 | TRUE |
| AU2 | NM | -6.3456 | -4.3958 | -2.4461 | TRUE |
| AU2 | FH | -4.2831 | -2.3333 | -0.3836 | TRUE |
| AU2 | RJ1 | -7.1894 | -5.2396 | -3.2898 | TRUE |
| AU2 | RJ2 | -8.7519 | -6.8021 | -4.8523 | TRUE |
| AU2 | XW1 | -9.1998 | -7.25 | -5.3002 | TRUE |
| AU2 | XW2 | -9.8873 | -7.9375 | -5.9877 | TRUE |
| AU3 | CG | -5.5956 | -3.6458 | -1.6961 | TRUE |
| AU3 | NM | -6.1789 | -4.2292 | -2.2794 | TRUE |
| AU3 | FH | -4.1164 | -2.1667 | -0.2169 | TRUE |
| AU3 | RJ1 | -7.0227 | -5.0729 | -3.1231 | TRUE |
| AU3 | RJ2 | -8.5852 | -6.6354 | -4.6856 | TRUE |
| AU3 | XW1 | -9.0331 | -7.0833 | -5.1336 | TRUE |
| AU3 | XW2 | -9.7206 | -7.7708 | -5.8211 | TRUE |
| CG | NM | -2.5331 | -0.5833 | 1.3664 | FALSE |
| CG | FH | -0.4706 | 1.4792 | 3.4289 | FALSE |
| CG | RJ1 | -3.3769 | -1.4271 | 0.5227 | FALSE |
| CG | RJ2 | -4.9394 | -2.9896 | -1.0398 | TRUE |
| CG | XW1 | -5.3873 | -3.4375 | -1.4877 | TRUE |
| CG | XW2 | -6.0748 | -4.125 | -2.1752 | TRUE |
| NM | FH | 0.1127 | 2.0625 | 4.0123 | TRUE |
| NM | RJ1 | -2.7935 | -0.8438 | 1.106 | FALSE |
| NM | RJ2 | -4.356 | -2.4063 | -0.4565 | TRUE |
| NM | XW1 | -4.8039 | -2.8542 | -0.9044 | TRUE |
| NM | XW2 | -5.4914 | -3.5417 | -1.5919 | TRUE |
| FH | RJ1 | -4.856 | -2.9063 | -0.9565 | TRUE |
| FH | RJ2 | -6.4185 | -4.4688 | -2.519 | TRUE |
| FH | XW1 | -6.8664 | -4.9167 | -2.9669 | TRUE |
| FH | XW2 | -7.5539 | -5.6042 | -3.6544 | TRUE |
| RJ1 | RJ2 | -3.5123 | -1.5625 | 0.3873 | FALSE |
| RJ1 | XW1 | -3.9602 | -2.0104 | -0.0606 | TRUE |
| RJ1 | XW2 | -4.6477 | -2.6979 | -0.7481 | TRUE |
| RJ2 | XW1 | -2.3977 | -0.4479 | 1.5019 | FALSE |
| RJ2 | XW2 | -3.0852 | -1.1354 | 0.8144 | FALSE |
| XW1 | XW2 | -2.6373 | -0.6875 | 1.2623 | FALSE |
Task 1 Summary Results by Query Group
| Group ID | Count | AU1 | AU2 | AU3 | CG | NM | FH | RJ1 | RJ2 | XW1 | XW2 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 9 | 0.023 | 0.010 | 0.006 | 0.423 | 0.726 | 0.347 | 0.722 | 0.944 | 1.000 | 1.000 |
| 2 | 9 | 0.059 | 0.120 | 0.117 | 0.598 | 0.578 | 0.056 | 0.778 | 0.806 | 0.444 | 0.455 |
| 3 | 8 | 0.500 | 0.250 | 0.188 | 1.000 | 0.700 | 0.438 | 0.875 | 0.875 | 0.875 | 0.875 |
| 4 | 9 | 0.519 | 0.444 | 0.472 | 0.786 | 0.901 | 0.467 | 1.000 | 1.000 | 0.944 | 1.000 |
| 5 | 9 | 0.333 | 0.222 | 0.229 | 0.309 | 0.494 | 0.278 | 0.083 | 0.561 | 0.917 | 1.000 |
| 6 | 8 | 0.188 | 0.000 | 0.031 | 0.380 | 0.563 | 0.080 | 0.760 | 1.000 | 1.000 | 1.000 |
| 7 | 9 | 0.236 | 0.111 | 0.111 | 0.346 | 0.722 | 0.287 | 0.889 | 0.944 | 0.897 | 0.944 |
| 8 | 9 | 0.160 | 0.111 | 0.111 | 0.401 | 0.615 | 0.228 | 0.800 | 0.562 | 1.000 | 1.000 |
| 9 | 8 | 0.196 | 0.375 | 0.375 | 0.688 | 0.608 | 0.138 | 0.503 | 1.000 | 1.000 | 1.000 |
| 10 | 10 | 0.426 | 0.064 | 0.032 | 0.684 | 0.810 | 0.660 | 1.000 | 0.825 | 1.000 | 1.000 |
| 11 | 62 | 0.349 | 0.142 | 0.194 | 0.541 | 0.627 | 0.519 | 0.685 | 0.922 | 0.922 | 0.929 |
| 12 | 99 | 0.352 | 0.119 | 0.212 | 0.684 | 0.701 | 0.498 | 0.838 | 0.926 | 0.932 | 0.934 |
| 13 | 105 | 0.111 | 0.068 | 0.094 | 0.302 | 0.640 | 0.343 | 0.810 | 0.922 | 0.946 | 0.956 |
| 14 | 99 | 0.378 | 0.170 | 0.235 | 0.643 | 0.661 | 0.347 | 0.740 | 0.907 | 0.923 | 0.940 |
| 15 | 15 | 0.157 | 0.101 | 0.185 | 0.401 | 0.561 | 0.229 | 0.636 | 0.794 | 0.833 | 0.826 |
| 16 | 101 | 0.493 | 0.173 | 0.226 | 0.529 | 0.683 | 0.568 | 0.703 | 0.874 | 0.924 | 0.940 |
| 17 | 95 | 0.099 | 0.060 | 0.045 | 0.224 | 0.425 | 0.065 | 0.410 | 0.787 | 0.942 | 0.961 |
| 18 | 104 | 0.244 | 0.173 | 0.211 | 0.562 | 0.657 | 0.362 | 0.709 | 0.913 | 0.988 | 0.995 |
| 19 | 104 | 0.328 | 0.107 | 0.131 | 0.653 | 0.651 | 0.440 | 0.876 | 0.956 | 0.938 | 0.954 |
| 20 | 108 | 0.140 | 0.062 | 0.133 | 0.318 | 0.457 | 0.210 | 0.753 | 0.943 | 0.959 | 0.972 |
| 21 | 46 | 0.182 | 0.022 | 0.018 | 0.438 | 0.433 | 0.263 | 0.487 | 0.649 | 0.655 | 0.665 |
| 22 | 108 | 0.232 | 0.104 | 0.160 | 0.560 | 0.797 | 0.407 | 0.850 | 0.916 | 0.987 | 0.986 |
| 23 | 45 | 0.394 | 0.044 | 0.000 | 0.676 | 0.480 | 0.478 | 0.703 | 0.961 | 0.944 | 0.956 |
| 24 | 75 | 0.142 | 0.029 | 0.062 | 0.146 | 0.347 | 0.135 | 0.645 | 0.821 | 0.773 | 0.785 |
| 25 | 77 | 0.169 | 0.019 | 0.015 | 0.392 | 0.553 | 0.318 | 0.205 | 0.499 | 0.685 | 0.708 |
| 26 | 39 | 0.372 | 0.088 | 0.024 | 0.487 | 0.533 | 0.309 | 0.768 | 0.893 | 0.885 | 0.897 |
| 27 | 52 | 0.206 | 0.019 | 0.020 | 0.705 | 0.429 | 0.395 | 0.808 | 0.971 | 0.971 | 0.971 |
| 28 | 55 | 0.149 | 0.011 | 0.010 | 0.189 | 0.067 | 0.225 | 0.352 | 0.519 | 0.809 | 0.839 |
| 29 | 97 | 0.195 | 0.071 | 0.075 | 0.256 | 0.378 | 0.200 | 0.824 | 0.860 | 0.944 | 0.958 |
| 30 | 87 | 0.331 | 0.187 | 0.216 | 0.401 | 0.640 | 0.374 | 0.802 | 0.933 | 0.908 | 0.943 |
| 31 | 90 | 0.217 | 0.135 | 0.143 | 0.677 | 0.651 | 0.387 | 0.899 | 0.942 | 0.969 | 1.000 |
| 32 | 85 | 0.246 | 0.058 | 0.015 | 0.545 | 0.702 | 0.499 | 0.431 | 0.891 | 0.951 | 0.975 |
| 33 | 76 | 0.135 | 0.018 | 0.007 | 0.399 | 0.423 | 0.318 | 0.526 | 0.781 | 0.653 | 0.722 |
| 34 | 89 | 0.181 | 0.044 | 0.041 | 0.272 | 0.529 | 0.393 | 0.633 | 0.856 | 0.900 | 0.934 |
| 35 | 93 | 0.111 | 0.057 | 0.056 | 0.340 | 0.447 | 0.234 | 0.715 | 0.946 | 0.893 | 0.916 |
| 36 | 25 | 0.354 | 0.141 | 0.214 | 0.377 | 0.811 | 0.415 | 0.920 | 0.920 | 0.920 | 0.960 |
| 37 | 39 | 0.237 | 0.139 | 0.130 | 0.575 | 0.452 | 0.431 | 0.600 | 0.811 | 0.932 | 0.949 |
| 38 | 68 | 0.189 | 0.102 | 0.014 | 0.756 | 0.492 | 0.450 | 0.883 | 0.891 | 0.925 | 0.912 |
| 39 | 95 | 0.318 | 0.088 | 0.133 | 0.610 | 0.677 | 0.400 | 0.773 | 0.864 | 0.906 | 0.891 |
| 40 | 52 | 0.181 | 0.067 | 0.103 | 0.320 | 0.461 | 0.292 | 0.716 | 0.837 | 0.877 | 0.904 |
| 41 | 77 | 0.209 | 0.109 | 0.135 | 0.310 | 0.702 | 0.434 | 0.694 | 0.882 | 0.961 | 0.956 |
| 42 | 60 | 0.391 | 0.133 | 0.138 | 0.626 | 0.685 | 0.433 | 0.810 | 0.929 | 0.919 | 0.953 |
| 43 | 73 | 0.276 | 0.131 | 0.172 | 0.621 | 0.676 | 0.386 | 0.565 | 0.923 | 0.948 | 0.970 |
| 44 | 48 | 0.340 | 0.062 | 0.035 | 0.567 | 0.720 | 0.493 | 0.739 | 0.928 | 0.958 | 0.958 |
| 45 | 65 | 0.088 | 0.066 | 0.066 | 0.425 | 0.662 | 0.283 | 0.716 | 0.878 | 0.901 | 0.928 |
| 46 | 49 | 0.226 | 0.020 | 0.005 | 0.550 | 0.545 | 0.459 | 0.757 | 0.893 | 0.949 | 0.963 |
| 47 | 20 | 0.115 | 0.000 | 0.013 | 0.491 | 0.663 | 0.224 | 0.821 | 0.925 | 0.967 | 0.967 |
| 48 | 32 | 0.254 | 0.110 | 0.096 | 0.586 | 0.394 | 0.243 | 0.868 | 0.880 | 0.956 | 0.940 |
Task 2 Results
In this subtask, the same setup as the first subtask used with combination of different transcribers and matchers. The test databases consists of 106 ground-truth MIDIS + 2000 Essen Collection MIDI noise files. The query databases consists of 355 sung queries.
Task 2 Legend
Team ID
FH_XW = Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine based on XW note transcriber
CG_XW = Carlos Gómez, Soraya Abad-Mota, Edna Ruckhaus based on XW note transcriber
RJ1_RJ = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on RJ pitch transcriber
RJ1_XW = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on XW pitch transcriber
RJ2_RJ = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on RJ pitch transcriber
RJ2_XW = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on XW pitch transcriber
NM_XW = Kjell Lemström, Niko Mikkilä based on XW note transcriber
XW1_XW = Xiao Wu, Ming Li 1 based on XW note transcriber
XW2_XW = Xiao Wu, Ming Li 2 based on XW pitch transcriber
Task 2 Overall Results
| Combinations | CG_XW | FH_XW | NM_XW | RJ1_RJ | RJ1_XW | RJ2_RJ | RJ2_XW | XW1_XW | XW2_XW | XW2_RJ |
|---|---|---|---|---|---|---|---|---|---|---|
| Task II (MRR) | 0.715 | 0.452 | 0.618 | 0.345 | 0.305 | 0.536 | 0.379 | 0.917 | 0.937 | 0.883 |
Task 2 Friedman's Test for Significant Differences
The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
| Friedman's ANOVA Table | |||||
|---|---|---|---|---|---|
| Source | SS | df | MS | Chi-sq | Prob>Chi-sq |
| Columns | 3.4149e+003 | 9 | 379.4329 | 451.1891 | 0 |
| Error | 3.8056e+003 | 945 | 4.0271 | ||
| Total | 7.2205e+003 | 1059 |
| ComboID | ComboID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| CG_XW | NM_XW | -0.3748 | 0.8208 | 2.0163 | FALSE |
| CG_XW | FH_XW | 0.7054 | 1.9009 | 3.0965 | TRUE |
| CG_XW | RJ1_RJ | 1.6582 | 2.8538 | 4.0493 | TRUE |
| CG_XW | RJ1_XW | 1.9365 | 3.1321 | 4.3276 | TRUE |
| CG_XW | RJ2_RJ | 0.4696 | 1.6651 | 2.8606 | TRUE |
| CG_XW | RJ2_XW | 1.3469 | 2.5425 | 3.738 | TRUE |
| CG_XW | XW1_XW | -2.7333 | -1.5377 | -0.3422 | TRUE |
| CG_XW | XW2_XW | -2.988 | -1.7925 | -0.5969 | TRUE |
| CG_XW | XW2_RJ | -2.5729 | -1.3774 | -0.1818 | TRUE |
| NM_XW | FH_XW | -0.1154 | 1.0802 | 2.2757 | FALSE |
| NM_XW | RJ1_RJ | 0.8375 | 2.033 | 3.2286 | TRUE |
| NM_XW | RJ1_XW | 1.1158 | 2.3113 | 3.5069 | TRUE |
| NM_XW | RJ2_RJ | -0.3512 | 0.8443 | 2.0399 | FALSE |
| NM_XW | RJ2_XW | 0.5262 | 1.7217 | 2.9172 | TRUE |
| NM_XW | XW1_XW | -3.554 | -2.3585 | -1.1629 | TRUE |
| NM_XW | XW2_XW | -3.8087 | -2.6132 | -1.4177 | TRUE |
| NM_XW | XW2_RJ | -3.3937 | -2.1981 | -1.0026 | TRUE |
| FH_XW | RJ1_RJ | -0.2427 | 0.9528 | 2.1484 | FALSE |
| FH_XW | RJ1_XW | 0.0356 | 1.2311 | 2.4267 | TRUE |
| FH_XW | RJ2_RJ | -1.4314 | -0.2358 | 0.9597 | FALSE |
| FH_XW | RJ2_XW | -0.554 | 0.6415 | 1.8371 | FALSE |
| FH_XW | XW1_XW | -4.6342 | -3.4387 | -2.2431 | TRUE |
| FH_XW | XW2_XW | -4.8889 | -3.6934 | -2.4979 | TRUE |
| FH_XW | XW2_RJ | -4.4738 | -3.2783 | -2.0828 | TRUE |
| RJ1_RJ | RJ1_XW | -0.9172 | 0.2783 | 1.4738 | FALSE |
| RJ1_RJ | RJ2_RJ | -2.3842 | -1.1887 | 0.0069 | FALSE |
| RJ1_RJ | RJ2_XW | -1.5069 | -0.3113 | 0.8842 | FALSE |
| RJ1_RJ | XW1_XW | -5.5871 | -4.3915 | -3.196 | TRUE |
| RJ1_RJ | XW2_XW | -5.8418 | -4.6462 | -3.4507 | TRUE |
| RJ1_RJ | XW2_RJ | -5.4267 | -4.2311 | -3.0356 | TRUE |
| RJ1_XW | RJ2_RJ | -2.6625 | -1.467 | -0.2714 | TRUE |
| RJ1_XW | RJ2_XW | -1.7852 | -0.5896 | 0.6059 | FALSE |
| RJ1_XW | XW1_XW | -5.8654 | -4.6698 | -3.4743 | TRUE |
| RJ1_XW | XW2_XW | -6.1201 | -4.9245 | -3.729 | TRUE |
| RJ1_XW | XW2_RJ | -5.705 | -4.5094 | -3.3139 | TRUE |
| RJ2_RJ | RJ2_XW | -0.3182 | 0.8774 | 2.0729 | FALSE |
| RJ2_RJ | XW1_XW | -4.3984 | -3.2028 | -2.0073 | TRUE |
| RJ2_RJ | XW2_XW | -4.6531 | -3.4575 | -2.262 | TRUE |
| RJ2_RJ | XW2_RJ | -4.238 | -3.0425 | -1.8469 | TRUE |
| RJ2_XW | XW1_XW | -5.2757 | -4.0802 | -2.8846 | TRUE |
| RJ2_XW | XW2_XW | -5.5304 | -4.3349 | -3.1394 | TRUE |
| RJ2_XW | XW2_RJ | -5.1154 | -3.9198 | -2.7243 | TRUE |
| XW1_XW | XW2_XW | -1.4503 | -0.2547 | 0.9408 | FALSE |
| XW1_XW | XW2_RJ | -1.0352 | 0.1604 | 1.3559 | FALSE |
| XW2_XW | XW2_RJ | -0.7804 | 0.4151 | 1.6106 | FALSE |
Task 2 Summary Results by Query Group
| Group ID | Count | CG_XW | NM_XW | FH_XW | RJ1_RJ | RJ1_XW | RJ2_RJ | RJ2_XW | XW1_XW | XW2_XW | XW2_RJ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 24 | 11 | 0.735 | 0.695 | 0.424 | 0.318 | 0.364 | 0.783 | 0.263 | 0.927 | 0.955 | 0.864 |
| 25 | 5 | 0.814 | 0.667 | 0.180 | 0.600 | 0.295 | 0.667 | 0.149 | 1.000 | 1.000 | 1.000 |
| 65 | 1 | 0.167 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.111 | 0.250 | 0.143 |
| 75 | 2 | 0.545 | 0.000 | 0.250 | 0.000 | 0.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 |
| 80 | 1 | 0.000 | 0.000 | 0.143 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.000 |
| 83 | 6 | 0.639 | 0.667 | 0.678 | 0.000 | 0.000 | 0.667 | 0.667 | 0.682 | 0.690 | 0.750 |
| 89 | 6 | 0.861 | 1.000 | 0.700 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 101 | 3 | 1.000 | 0.030 | 0.000 | 0.056 | 0.048 | 0.017 | 0.000 | 1.000 | 1.000 | 1.000 |
| 106 | 5 | 0.733 | 0.411 | 0.200 | 0.000 | 0.000 | 0.233 | 0.050 | 1.000 | 1.000 | 1.000 |
| 108 | 2 | 0.750 | 1.000 | 0.000 | 0.000 | 0.000 | 0.031 | 0.071 | 1.000 | 1.000 | 1.000 |
| 110 | 2 | 0.417 | 0.063 | 0.029 | 0.000 | 0.000 | 0.050 | 0.083 | 0.625 | 0.750 | 1.000 |
| 111 | 1 | 1.000 | 1.000 | 0.500 | 1.000 | 1.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 |
| 114 | 5 | 0.458 | 0.450 | 0.022 | 0.000 | 0.000 | 0.400 | 0.300 | 1.000 | 1.000 | 0.800 |
| 116 | 2 | 0.600 | 0.500 | 0.250 | 0.000 | 0.000 | 0.500 | 0.125 | 1.000 | 1.000 | 1.000 |
| 120 | 2 | 0.500 | 0.000 | 0.000 | 0.000 | 0.000 | 0.500 | 0.528 | 0.600 | 1.000 | 0.750 |
| 128 | 4 | 0.400 | 0.642 | 0.516 | 0.156 | 0.000 | 0.053 | 0.287 | 1.000 | 1.000 | 1.000 |
| 129 | 1 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.333 |
| 133 | 2 | 0.500 | 1.000 | 0.500 | 0.000 | 0.500 | 0.600 | 1.000 | 1.000 | 1.000 | 1.000 |
| 139 | 2 | 1.000 | 0.667 | 0.313 | 0.000 | 0.000 | 1.000 | 0.625 | 1.000 | 1.000 | 1.000 |
| 140 | 2 | 0.375 | 0.042 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.333 | 0.417 | 0.500 |
| 160 | 2 | 0.750 | 0.750 | 0.500 | 0.061 | 0.036 | 0.250 | 0.125 | 0.525 | 0.550 | 0.500 |
| 162 | 9 | 0.790 | 0.674 | 0.365 | 0.444 | 0.278 | 0.911 | 0.590 | 0.944 | 0.917 | 0.787 |
| 167 | 1 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 185 | 2 | 1.000 | 1.000 | 0.750 | 0.000 | 0.000 | 1.000 | 0.500 | 1.000 | 1.000 | 1.000 |
| 192 | 2 | 1.000 | 1.000 | 0.533 | 0.083 | 0.100 | 0.500 | 0.500 | 1.000 | 1.000 | 0.500 |
| 193 | 11 | 0.740 | 0.778 | 0.327 | 0.365 | 0.286 | 0.360 | 0.291 | 1.000 | 1.000 | 1.000 |
| 195 | 4 | 0.750 | 1.000 | 0.750 | 0.250 | 0.250 | 0.453 | 0.750 | 1.000 | 1.000 | 1.000 |
| 197 | 4 | 0.625 | 0.500 | 0.125 | 0.500 | 0.161 | 0.750 | 0.500 | 1.000 | 1.000 | 1.000 |
| 210 | 1 | 1.000 | 0.000 | 0.000 | 0.250 | 0.250 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 |
| 232 | 4 | 1.000 | 0.633 | 0.750 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 233 | 1 | 1.000 | 1.000 | 0.000 | 1.000 | 1.000 | 1.000 | 0.167 | 1.000 | 1.000 | 1.000 |
| 245 | 7 | 0.714 | 0.679 | 0.571 | 0.000 | 0.000 | 0.786 | 0.378 | 1.000 | 1.000 | 1.000 |
| 250 | 5 | 0.700 | 1.000 | 1.000 | 0.000 | 0.000 | 0.800 | 0.511 | 0.662 | 0.718 | 0.414 |
| 265 | 2 | 0.500 | 0.000 | 0.571 | 0.000 | 0.000 | 0.500 | 0.083 | 0.500 | 0.500 | 0.500 |
| 271 | 1 | 1.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 |
| 276 | 5 | 1.000 | 0.247 | 0.200 | 0.000 | 0.000 | 0.425 | 0.350 | 0.800 | 0.867 | 0.800 |
| 278 | 1 | 0.000 | 0.000 | 0.091 | 0.000 | 0.000 | 1.000 | 0.000 | 0.000 | 0.063 | 0.000 |
| 279 | 2 | 1.000 | 0.500 | 0.500 | 0.500 | 0.500 | 0.500 | 0.500 | 1.000 | 1.000 | 1.000 |
| 289 | 7 | 0.524 | 0.310 | 0.171 | 0.429 | 0.333 | 0.722 | 0.361 | 1.000 | 1.000 | 1.000 |
| 293 | 2 | 0.750 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.625 | 1.000 | 1.000 | 0.000 |
| 306 | 8 | 0.604 | 0.813 | 0.639 | 0.250 | 0.250 | 0.427 | 0.046 | 0.875 | 0.875 | 0.875 |
| 319 | 12 | 0.651 | 0.521 | 0.613 | 0.625 | 0.508 | 0.325 | 0.121 | 0.875 | 1.000 | 0.917 |
| 321 | 9 | 0.411 | 0.376 | 0.175 | 0.593 | 0.358 | 0.458 | 0.186 | 0.763 | 0.833 | 0.903 |
| 362 | 9 | 0.750 | 0.704 | 0.306 | 0.556 | 0.398 | 0.926 | 0.648 | 1.000 | 1.000 | 1.000 |
| 363 | 3 | 0.667 | 0.381 | 0.000 | 1.000 | 0.833 | 0.667 | 0.667 | 1.000 | 1.000 | 1.000 |
| 392 | 4 | 0.583 | 0.250 | 0.250 | 0.031 | 0.000 | 0.268 | 0.250 | 1.000 | 1.000 | 0.750 |
| 399 | 4 | 0.781 | 0.375 | 0.396 | 0.313 | 0.119 | 0.467 | 0.018 | 1.000 | 1.000 | 1.000 |
| 438 | 1 | 1.000 | 1.000 | 1.000 | 1.000 | 0.071 | 0.000 | 0.067 | 1.000 | 1.000 | 1.000 |
| 461 | 1 | 1.000 | 1.000 | 0.000 | 1.000 | 1.000 | 1.000 | 0.333 | 1.000 | 1.000 | 1.000 |
| 468 | 1 | 1.000 | 1.000 | 0.125 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 475 | 4 | 0.583 | 0.750 | 1.000 | 0.250 | 0.125 | 0.625 | 0.375 | 1.000 | 1.000 | 1.000 |
| 513 | 4 | 0.411 | 0.088 | 0.250 | 0.125 | 0.000 | 0.290 | 0.333 | 1.000 | 1.000 | 1.000 |
| 514 | 6 | 0.679 | 0.649 | 0.556 | 0.333 | 0.192 | 1.000 | 0.408 | 1.000 | 1.000 | 1.000 |
| 515 | 6 | 0.639 | 0.667 | 0.167 | 0.308 | 0.431 | 0.392 | 0.722 | 1.000 | 1.000 | 1.000 |
| 529 | 4 | 0.769 | 0.750 | 0.500 | 0.000 | 0.000 | 0.000 | 0.013 | 1.000 | 1.000 | 1.000 |
| 533 | 2 | 0.500 | 0.500 | 0.500 | 0.500 | 0.500 | 0.500 | 0.500 | 0.500 | 0.500 | 0.000 |
| 540 | 5 | 0.712 | 1.000 | 0.707 | 0.767 | 1.000 | 0.667 | 0.850 | 1.000 | 1.000 | 0.500 |
| 550 | 1 | 1.000 | 1.000 | 0.083 | 0.000 | 0.000 | 0.067 | 0.167 | 1.000 | 1.000 | 1.000 |
| 561 | 3 | 0.511 | 1.000 | 0.733 | 0.000 | 0.000 | 1.000 | 0.778 | 1.000 | 1.000 | 1.000 |
| 562 | 2 | 0.417 | 0.110 | 0.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 568 | 1 | 1.000 | 1.000 | 1.000 | 0.000 | 0.000 | 0.167 | 0.500 | 1.000 | 1.000 | 1.000 |
| 570 | 3 | 0.833 | 0.667 | 0.667 | 0.037 | 0.067 | 0.444 | 0.377 | 1.000 | 1.000 | 1.000 |
| 572 | 2 | 0.600 | 0.500 | 1.000 | 1.000 | 0.750 | 0.750 | 0.500 | 1.000 | 1.000 | 1.000 |
| 573 | 1 | 0.125 | 1.000 | 1.000 | 0.000 | 0.000 | 0.333 | 0.250 | 1.000 | 1.000 | 1.000 |
| 594 | 3 | 1.000 | 0.500 | 0.000 | 0.000 | 0.000 | 0.364 | 0.208 | 1.000 | 1.000 | 1.000 |
| 606 | 4 | 1.000 | 0.769 | 0.750 | 0.333 | 0.500 | 0.542 | 0.750 | 1.000 | 1.000 | 0.750 |
| 628 | 3 | 0.667 | 0.833 | 0.394 | 0.000 | 0.000 | 1.000 | 0.167 | 1.000 | 1.000 | 1.000 |
| 636 | 2 | 0.250 | 1.000 | 0.025 | 1.000 | 0.288 | 0.550 | 0.000 | 0.667 | 0.667 | 1.000 |
| 643 | 4 | 0.875 | 0.750 | 0.125 | 0.050 | 0.013 | 0.500 | 0.300 | 1.000 | 1.000 | 1.000 |
| 647 | 4 | 1.000 | 0.786 | 0.641 | 0.875 | 0.833 | 0.606 | 0.750 | 1.000 | 1.000 | 1.000 |
| 660 | 3 | 0.500 | 1.000 | 0.833 | 0.000 | 0.000 | 0.370 | 0.167 | 0.421 | 0.417 | 0.667 |
| 662 | 3 | 0.833 | 0.367 | 0.667 | 1.000 | 1.000 | 0.567 | 0.567 | 1.000 | 1.000 | 1.000 |
| 685 | 4 | 0.750 | 1.000 | 1.000 | 1.000 | 0.875 | 0.750 | 0.354 | 1.000 | 1.000 | 1.000 |
| 693 | 2 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.045 | 0.167 | 1.000 | 1.000 | 1.000 |
| 728 | 3 | 0.778 | 0.333 | 0.208 | 0.333 | 0.333 | 0.333 | 0.361 | 1.000 | 1.000 | 1.000 |
| 730 | 2 | 1.000 | 0.500 | 0.276 | 0.000 | 0.000 | 0.417 | 0.050 | 0.550 | 1.000 | 0.500 |
| 732 | 10 | 0.870 | 0.900 | 1.000 | 1.000 | 0.950 | 0.950 | 0.491 | 1.000 | 1.000 | 1.000 |
| 736 | 1 | 1.000 | 0.500 | 1.000 | 0.056 | 1.000 | 0.111 | 0.500 | 1.000 | 1.000 | 1.000 |
| 747 | 2 | 1.000 | 1.000 | 0.500 | 0.500 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 770 | 3 | 1.000 | 1.000 | 0.750 | 0.333 | 0.333 | 0.750 | 0.444 | 1.000 | 1.000 | 1.000 |
| 790 | 6 | 1.000 | 0.769 | 0.690 | 0.250 | 0.333 | 0.696 | 0.271 | 0.917 | 1.000 | 0.667 |
| 794 | 2 | 1.000 | 1.000 | 1.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 805 | 3 | 0.333 | 0.333 | 0.186 | 0.000 | 0.000 | 0.148 | 0.111 | 1.000 | 1.000 | 0.667 |
| 811 | 4 | 0.377 | 0.000 | 0.046 | 0.000 | 0.000 | 0.000 | 0.000 | 0.750 | 0.750 | 0.000 |
| 839 | 4 | 0.750 | 1.000 | 0.181 | 0.500 | 0.500 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 850 | 1 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 858 | 3 | 1.000 | 1.000 | 1.000 | 0.000 | 0.000 | 1.000 | 0.483 | 1.000 | 1.000 | 1.000 |
| 914 | 1 | 1.000 | 1.000 | 0.333 | 0.000 | 0.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 |
| 915 | 7 | 0.929 | 0.729 | 0.786 | 0.143 | 0.143 | 0.143 | 0.286 | 0.857 | 0.857 | 0.905 |
| 926 | 2 | 1.000 | 0.500 | 0.500 | 0.000 | 0.000 | 0.000 | 0.026 | 1.000 | 1.000 | 1.000 |
| 937 | 2 | 0.600 | 0.500 | 0.083 | 0.000 | 0.000 | 0.250 | 0.500 | 1.000 | 1.000 | 1.000 |
| 941 | 2 | 0.500 | 0.538 | 1.000 | 0.500 | 0.500 | 0.167 | 0.071 | 1.000 | 1.000 | 1.000 |
| 956 | 2 | 1.000 | 0.071 | 0.250 | 0.031 | 0.100 | 0.250 | 0.050 | 1.000 | 1.000 | 1.000 |
| 1000 | 1 | 0.500 | 0.500 | 1.000 | 0.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 1014 | 1 | 1.000 | 0.500 | 1.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| 1036 | 1 | 0.000 | 0.000 | 0.000 | 0.056 | 0.071 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 |
| 1039 | 1 | 1.000 | 1.000 | 1.000 | 0.000 | 0.000 | 0.077 | 0.111 | 1.000 | 1.000 | 1.000 |
| 1044 | 1 | 1.000 | 1.000 | 0.333 | 0.000 | 0.000 | 0.167 | 0.000 | 1.000 | 1.000 | 1.000 |
| 1054 | 4 | 0.653 | 0.271 | 0.313 | 0.583 | 0.750 | 0.000 | 0.063 | 0.875 | 1.000 | 1.000 |
| 1062 | 3 | 1.000 | 0.750 | 0.356 | 0.333 | 0.333 | 0.108 | 0.111 | 1.000 | 1.000 | 1.000 |
| 1067 | 1 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
| 1068 | 1 | 0.250 | 1.000 | 1.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 |
| 1070 | 8 | 0.592 | 0.400 | 0.094 | 0.375 | 0.229 | 0.700 | 0.455 | 1.000 | 1.000 | 0.750 |
| 1082 | 3 | 0.833 | 0.300 | 0.083 | 0.000 | 0.000 | 0.733 | 0.667 | 1.000 | 1.000 | 1.000 |
| 1083 | 1 | 0.200 | 0.143 | 0.000 | 1.000 | 0.500 | 0.000 | 0.000 | 0.053 | 0.143 | 0.500 |
| 1094 | 3 | 0.500 | 0.061 | 0.000 | 0.000 | 0.000 | 0.047 | 0.048 | 0.694 | 1.000 | 1.000 |
