Difference between revisions of "2015:Music/Speech Classification and Detection Results"
Emmanouilb (talk | contribs) (→General Legend) |
Emmanouilb (talk | contribs) (→Introduction) |
||
(44 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
− | These are the results for the 2015 running of the Music/Speech Classification and Detection task. For background information about this task set please refer to the [[2015:Music/ | + | These are the results for the 2015 running of the Music/Speech Classification and Detection task. For background information about this task set please refer to the [[2015:Music/Speech Classification and Detection]] page. |
==General Legend== | ==General Legend== | ||
Line 31: | Line 31: | ||
|- | |- | ||
! RS2 | ! RS2 | ||
− | | PDF || Reinhard Sonnleitner | + | | [https://www.music-ir.org/mirex/abstracts/2015/RS2.pdf PDF] || Reinhard Sonnleitner |
|- | |- | ||
! TL1 | ! TL1 | ||
Line 71: | Line 71: | ||
! NT2 | ! NT2 | ||
| [https://www.music-ir.org/mirex/abstracts/2015/NT2.pdf PDF] || Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou | | [https://www.music-ir.org/mirex/abstracts/2015/NT2.pdf PDF] || Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou | ||
+ | |- | ||
+ | ! YZ1 | ||
+ | | [https://www.music-ir.org/mirex/abstracts/2015/YZ1.pdf PDF] || Wei-Qiang Zhang, Xu-Kui Yang | ||
+ | |} | ||
+ | |||
+ | ==Task 1: Music/Speech Classification== | ||
+ | |||
+ | '''Complete dataset''' | ||
+ | |||
+ | {| border="1" cellspacing="0" style="text-align: left; width: 240px;" | ||
+ | |- style="background: yellow;" | ||
+ | ! width="80" | Sub code | ||
+ | ! width="80" style="text-align: center;" | Accuracy | ||
+ | ! width="80" | std | ||
+ | |- | ||
+ | ! GWDS1 | ||
+ | | 0.9866 || 0.0065 | ||
+ | |- | ||
+ | ! JS3 | ||
+ | | 0.9946 || 0.0027 | ||
+ | |- | ||
+ | ! MM1 | ||
+ | | 0.9869 || 0.0037 | ||
+ | |- | ||
+ | ! MM2 | ||
+ | | 0.9754 || 0.007 | ||
+ | |- | ||
+ | ! RS2 | ||
+ | | 0.9962 || 0.0029 | ||
+ | |- | ||
+ | ! TL1(15) | ||
+ | | 0.9927 || 0.0018 | ||
+ | |- | ||
+ | ! TL1(80) | ||
+ | | 0.9973 || 0.0018 | ||
+ | |- | ||
+ | ! ZC4 | ||
+ | | 0.9429 || 0.0076 | ||
+ | |- | ||
+ | ! ZC5 | ||
+ | | 0.9429 || 0.0076 | ||
+ | |- | ||
+ | ! NT1 | ||
+ | | 0.9835 || 0.0109 | ||
+ | |- | ||
+ | ! NT2 | ||
+ | | 0.9935 || 0.0044 | ||
+ | |- | ||
+ | ! RHM1 | ||
+ | | 0.9954 || 0.004 | ||
+ | |- | ||
+ | ! YZ1 | ||
+ | | 0.9835 || 0.0044 | ||
+ | |} | ||
+ | |||
+ | ===Individual Results Files for Task 1=== | ||
+ | |||
+ | '''GWDS1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/GWDS1_confusion_matrix.csv Vikaskumar Ghodasara, Daimi Syed Naser, Shefali Waldekar, Goutam Saha ]<br /> | ||
+ | '''JS3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/JS3_confusion_matrix.csv Jan Schlüter ]<br /> | ||
+ | '''MM1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/MM1_confusion_matrix.csv Matija Marolt ]<br /> | ||
+ | '''MM2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/MM2_confusion_matrix.csv Matija Marolt ]<br /> | ||
+ | '''RS2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/RS2_confusion_matrix.csv Reinhard Sonnleitner ]<br /> | ||
+ | '''TL1(15)'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/TL1_confusion_matrix.csv Thomas Lidy ]<br /> | ||
+ | '''TL1(80)'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/TL1-80_confusion_matrix.csv Thomas Lidy ]<br /> | ||
+ | '''ZC4'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/ZC4_confusion_matrix.csv Chaogang Zhang, Chuan-Yi Chen ]<br /> | ||
+ | '''ZC5'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/ZC5_confusion_matrix.csv Chaogang Zhang, Chuan-Yi Chen ]<br /> | ||
+ | '''NT1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/NT1_confusion_matrix.csv Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br /> | ||
+ | '''NT2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/NT2_confusion_matrix.csv Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br /> | ||
+ | '''RHM1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/RHM1_confusion_matrix.csv Jimena Royo-Letelier, Romain Hennequin, Manuel Moussallam ]<br /> | ||
+ | '''YZ1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/YZ1_confusion_matrix.csv Wei-Qiang Zhang, Xu-Kui Yang ]<br /> | ||
+ | |||
+ | ==Task 2: Music/Speech Detection== | ||
+ | |||
+ | {| border="1" cellspacing="0" style="text-align: left; width: 600px;" | ||
+ | |- style="background: yellow;" | ||
+ | ! width="80" | Sub code | ||
+ | ! width="80" | fb_F | ||
+ | ! width="80" | eb_F_500ms | ||
+ | ! width="80" | eb_Foff_500ms | ||
+ | ! width="80" | eb_F_1s | ||
+ | ! width="80" | eb_Foff_1s | ||
+ | |- | ||
+ | ! JS2 | ||
+ | | 0.8233 || 0.1754 || 0.1212 || 0.2387 || 0.1639 | ||
+ | |- | ||
+ | ! MM3 | ||
+ | | 0.8941 || 0.4037 || 0.2911 || 0.4419 || 0.3085 | ||
+ | |- | ||
+ | ! TVDP1 | ||
+ | | 0.8314 || 0.1490 || 0.1146 || 0.2233 || 0.1718 | ||
+ | |- | ||
+ | ! TVDP2 | ||
+ | | 0.8346 || 0.1250 || 0.1231 || 0.2042 || 0.1741 | ||
+ | |- | ||
+ | ! TVDP3 | ||
+ | | 0.8209 || 0.0699 || 0.0068 || 0.1299 || 0.0306 | ||
+ | |- | ||
+ | ! UK1 | ||
+ | | 0.5483 || 0.0324 || 0.0037 || 0.0623 || 0.0070 | ||
+ | |- | ||
+ | ! ZC1 | ||
+ | | 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101 | ||
+ | |- | ||
+ | ! ZC2 | ||
+ | | 0.2079 || 0.0434 || 0.0434 || 0.0434 || 0.0434 | ||
+ | |- | ||
+ | ! ZC3 | ||
+ | | 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101 | ||
+ | |- | ||
+ | ! ZC4 | ||
+ | | 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101 | ||
+ | |- | ||
+ | ! ZC5 | ||
+ | | 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101 | ||
+ | |- | ||
+ | ! ZY1 | ||
+ | | 0.5104 || 0.1670 || 0.0874 || 0.2164 || 0.1177 | ||
+ | |- | ||
+ | ! TL1 | ||
+ | | 0.8849 || 0.2129 || 0.1129 || 0.2556 || 0.1498 | ||
|} | |} | ||
+ | |||
+ | '''Notes on metrics:''' | ||
+ | |||
+ | fb_F=frame-based F-measure | ||
+ | |||
+ | eb_F_500ms=onset-only event-based F-measure (500ms tolerance) | ||
+ | |||
+ | eb_Foff_500ms=onset-offset event-based F-measure (500ms tolerance) | ||
+ | |||
+ | eb_F_1s=onset-only event-based F-measure (1s tolerance) | ||
+ | |||
+ | eb_Foff_1s=onset-offset event-based F-measure (1s tolerance) | ||
+ | |||
+ | |||
+ | ===Individual Results Files for Task 2=== | ||
+ | |||
+ | '''JS2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_JS2.mat Jan Schlüter ]<br /> | ||
+ | '''MM3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_MM3.mat Matija Marolt ]<br /> | ||
+ | '''TVDP1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TVDP1.mat Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br /> | ||
+ | '''TVDP2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TVDP2.mat Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br /> | ||
+ | '''TVDP3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TVDP3.mat Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br /> | ||
+ | '''UK1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_UK1.mat Aiko Uemura, Jiro Katto ]<br /> | ||
+ | '''ZC1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC1.mat Chaogang Zhang, Chuan-Yi Chen ]<br /> | ||
+ | '''ZC2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC2.mat Chaogang Zhang, Chuan-Yi Chen ]<br /> | ||
+ | '''ZC3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC3.mat Chaogang Zhang, Chuan-Yi Chen ]<br /> | ||
+ | '''ZC4'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC4.mat Chaogang Zhang, Chuan-Yi Chen ]<br /> | ||
+ | '''ZC5'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC5.mat Chaogang Zhang, Chuan-Yi Chen ]<br /> | ||
+ | '''ZY1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZY1.mat Wei-Qiang Zhang, Xu-Kui Yang ]<br /> | ||
+ | '''TL1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TL1-80.mat Thomas Lidy ] (TBA) <br /> | ||
+ | |||
+ | Note: detailed results are stored as binary Matlab files (.mat) |
Latest revision as of 08:20, 25 February 2016
Contents
Introduction
These are the results for the 2015 running of the Music/Speech Classification and Detection task. For background information about this task set please refer to the 2015:Music/Speech Classification and Detection page.
General Legend
Sub code | Abstract | Contributors |
---|---|---|
GWDS1 | Vikaskumar Ghodasara, Daimi Syed Naser, Shefali Waldekar, Goutam Saha | |
JS2 | Jan Schlüter | |
JS3 | Jan Schlüter | |
MM1 | Matija Marolt | |
MM2 | Matija Marolt | |
MM3 | Matija Marolt | |
RHM1 | Jimena Royo-Letelier, Romain Hennequin, Manuel Moussallam | |
RS2 | Reinhard Sonnleitner | |
TL1 | Thomas Lidy | |
UK1 | Aiko Uemura, Jiro Katto | |
ZC1 | Chaogang Zhang, Chuan-Yi Chen | |
ZC2 | Chaogang Zhang, Chuan-Yi Chen | |
ZC3 | Chaogang Zhang, Chuan-Yi Chen | |
ZC4 | Chaogang Zhang, Chuan-Yi Chen | |
ZC5 | Chaogang Zhang, Chuan-Yi Chen | |
ZY1 | Wei-Qiang Zhang, Xu-Kui Yang | |
TVDP1 | Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou | |
TVDP2 | Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou | |
TVDP3 | Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou | |
NT1 | Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou | |
NT2 | Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou | |
YZ1 | Wei-Qiang Zhang, Xu-Kui Yang |
Task 1: Music/Speech Classification
Complete dataset
Sub code | Accuracy | std |
---|---|---|
GWDS1 | 0.9866 | 0.0065 |
JS3 | 0.9946 | 0.0027 |
MM1 | 0.9869 | 0.0037 |
MM2 | 0.9754 | 0.007 |
RS2 | 0.9962 | 0.0029 |
TL1(15) | 0.9927 | 0.0018 |
TL1(80) | 0.9973 | 0.0018 |
ZC4 | 0.9429 | 0.0076 |
ZC5 | 0.9429 | 0.0076 |
NT1 | 0.9835 | 0.0109 |
NT2 | 0.9935 | 0.0044 |
RHM1 | 0.9954 | 0.004 |
YZ1 | 0.9835 | 0.0044 |
Individual Results Files for Task 1
GWDS1= Vikaskumar Ghodasara, Daimi Syed Naser, Shefali Waldekar, Goutam Saha
JS3= Jan Schlüter
MM1= Matija Marolt
MM2= Matija Marolt
RS2= Reinhard Sonnleitner
TL1(15)= Thomas Lidy
TL1(80)= Thomas Lidy
ZC4= Chaogang Zhang, Chuan-Yi Chen
ZC5= Chaogang Zhang, Chuan-Yi Chen
NT1= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
NT2= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
RHM1= Jimena Royo-Letelier, Romain Hennequin, Manuel Moussallam
YZ1= Wei-Qiang Zhang, Xu-Kui Yang
Task 2: Music/Speech Detection
Sub code | fb_F | eb_F_500ms | eb_Foff_500ms | eb_F_1s | eb_Foff_1s |
---|---|---|---|---|---|
JS2 | 0.8233 | 0.1754 | 0.1212 | 0.2387 | 0.1639 |
MM3 | 0.8941 | 0.4037 | 0.2911 | 0.4419 | 0.3085 |
TVDP1 | 0.8314 | 0.1490 | 0.1146 | 0.2233 | 0.1718 |
TVDP2 | 0.8346 | 0.1250 | 0.1231 | 0.2042 | 0.1741 |
TVDP3 | 0.8209 | 0.0699 | 0.0068 | 0.1299 | 0.0306 |
UK1 | 0.5483 | 0.0324 | 0.0037 | 0.0623 | 0.0070 |
ZC1 | 0.7517 | 0.2168 | 0.1956 | 0.2700 | 0.2101 |
ZC2 | 0.2079 | 0.0434 | 0.0434 | 0.0434 | 0.0434 |
ZC3 | 0.7517 | 0.2168 | 0.1956 | 0.2700 | 0.2101 |
ZC4 | 0.7517 | 0.2168 | 0.1956 | 0.2700 | 0.2101 |
ZC5 | 0.7517 | 0.2168 | 0.1956 | 0.2700 | 0.2101 |
ZY1 | 0.5104 | 0.1670 | 0.0874 | 0.2164 | 0.1177 |
TL1 | 0.8849 | 0.2129 | 0.1129 | 0.2556 | 0.1498 |
Notes on metrics:
fb_F=frame-based F-measure
eb_F_500ms=onset-only event-based F-measure (500ms tolerance)
eb_Foff_500ms=onset-offset event-based F-measure (500ms tolerance)
eb_F_1s=onset-only event-based F-measure (1s tolerance)
eb_Foff_1s=onset-offset event-based F-measure (1s tolerance)
Individual Results Files for Task 2
JS2= Jan Schlüter
MM3= Matija Marolt
TVDP1= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
TVDP2= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
TVDP3= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
UK1= Aiko Uemura, Jiro Katto
ZC1= Chaogang Zhang, Chuan-Yi Chen
ZC2= Chaogang Zhang, Chuan-Yi Chen
ZC3= Chaogang Zhang, Chuan-Yi Chen
ZC4= Chaogang Zhang, Chuan-Yi Chen
ZC5= Chaogang Zhang, Chuan-Yi Chen
ZY1= Wei-Qiang Zhang, Xu-Kui Yang
TL1= Thomas Lidy (TBA)
Note: detailed results are stored as binary Matlab files (.mat)