Difference between revisions of "2011:Query-by-Singing/Humming Results"

From MIREX Wiki
(Task 1a Overall Results)
(Task 2 Summary Results by Query)
Line 137: Line 137:
  
 
====Task 2 Summary Results by Query ====
 
====Task 2 Summary Results by Query ====
[https://www.music-ir.org/mirex/results/2011/qbsh/QbshTask2Query.csv]
+
<csv>2011/qbsh/QbshTask2Query.csv</csv>
  
 
===Runtime Results===
 
===Runtime Results===

Revision as of 21:46, 21 October 2011

Introduction

These are the results for the 2008 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the 2011:Query by Singing/Humming page.

Task Descriptions

Task 1 Goto Task 1 Results: The first subtask is the same as last year. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth, as well as the simple hit(1)/miss(0) counting, is calculated over the top 10 returns. Two data sets are used:

Task 2 Goto Task 2 Results: The second subtask is the query against other humming. In the second subtask, Roger Jang's MIR-QBSH corpus has been divided into two groups (2040 as queries and 2391 as database). The query is performed against the other humming database and the top 10 closed are returned. The score is simple count of how many returns belong to the same ground truth song.

General Legend

General Legend

Sub code Submission name Abstract Contributors
TY1 QBSH algorithm 1 PDF Tzu-Chun Yeh
TY2 QBSH algorithm 2 PDF Tzu-Chun Yeh
JSSLP1 KETI PDF Dalwon Jang, SUNG JOO PARK,Chai-Jong Song,Saim Shin,SEI-JIN JANG,Jong-Seol Lee,Seok-Pil Lee,Kyeung Hak Seo


Task 1 Results

Task 1a, Jang's dataset Results

Task 1a Overall Results
JSSLP1 TY1 TY2
Simple Count 0.939 0.956 0.919
MRR 0.897 0.93 0.881
Total Count 4431 4431 4431

download these results as csv

Task 1a Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1a Simple/MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID TeamID Lowerbound Mean Upperbound Significance
TY1 JSSLP1 -0.0749 0.3438 0.7624 FALSE
TY1 TY2 0.6439 1.0625 1.4811 TRUE
JSSLP1 TY2 0.3001 0.7188 1.1374 TRUE

download these results as csv

2011QbshTask1aSimpleByGroup.friedman.tukeyKramerHSD.png

MRR Method:

TeamID TeamID Lowerbound Mean Upperbound Significance
TY1 JSSLP1 0.2321 0.6979 1.1637 TRUE
TY1 TY2 0.7426 1.2083 1.6741 TRUE
JSSLP1 TY2 0.0446 0.5104 0.9762 TRUE

download these results as csv

2011QbshTask1aMrrByGroup.friedman.tukeyKramerHSD.png

Task 1a Summary Results by Query Group

Simple Hit/Miss Count

JSSLP1 TY1 TY2
1 0.947 0.947 0.895
2 0.714 0.786 0.643
3 0.727 0.727 0.727
4 0.8 0.8 0.8
5 0.933 0.933 0.8
6 0.833 0.833 0.917
7 0.867 0.933 0.867
8 0.933 0.933 0.933
9 0.917 0.917 0.917
10 0.947 0.684 0.842
11 0.927 0.948 0.896
12 0.968 0.968 0.961
13 0.95 0.981 0.963
14 0.975 0.994 0.956
15 0.963 0.963 0.963
16 0.931 0.969 0.925
17 0.941 0.98 0.928
18 0.97 0.976 0.957
19 0.949 0.968 0.943
20 0.971 1 0.982
21 0.742 0.818 0.758
22 0.977 0.988 0.977
23 1 0.984 0.952
24 0.752 0.843 0.785
25 0.756 0.862 0.715
26 0.962 0.962 0.962
27 0.965 0.965 0.965
28 0.879 0.879 0.725
29 0.993 0.987 0.973
30 0.959 0.986 0.938
31 0.979 0.993 0.972
32 0.964 0.956 0.934
33 0.94 0.94 0.862
34 0.949 0.927 0.891
35 0.926 0.966 0.886
36 0.953 0.93 0.953
37 0.939 0.939 0.848
38 0.965 0.93 0.956
39 0.952 0.98 0.939
40 0.93 0.972 0.915
41 0.944 0.984 0.913
42 0.957 0.946 0.946
43 0.951 0.976 0.943
44 0.95 0.95 0.95
45 0.97 0.97 0.939
46 0.989 0.978 0.935
47 0.931 1 0.931
48 0.962 1 0.925

download these results as csv

MRR Method

JSSLP1 TY1 TY2
1 0.947 0.901 0.847
2 0.714 0.732 0.595
3 0.682 0.682 0.645
4 0.8 0.8 0.8
5 0.933 0.85 0.767
6 0.833 0.833 0.808
7 0.867 0.889 0.867
8 0.933 0.88 0.933
9 0.917 0.917 0.844
10 0.947 0.61 0.842
11 0.876 0.932 0.868
12 0.945 0.958 0.937
13 0.917 0.94 0.938
14 0.865 0.969 0.879
15 0.944 0.907 0.93
16 0.889 0.922 0.891
17 0.905 0.95 0.857
18 0.959 0.965 0.947
19 0.931 0.968 0.925
20 0.929 0.977 0.944
21 0.632 0.722 0.678
22 0.973 0.984 0.966
23 0.965 0.962 0.939
24 0.688 0.78 0.694
25 0.541 0.79 0.595
26 0.942 0.946 0.939
27 0.928 0.965 0.951
28 0.776 0.798 0.597
29 0.953 0.968 0.959
30 0.934 0.969 0.91
31 0.965 0.988 0.944
32 0.93 0.956 0.891
33 0.886 0.92 0.802
34 0.882 0.908 0.85
35 0.887 0.948 0.876
36 0.935 0.93 0.953
37 0.908 0.903 0.833
38 0.949 0.906 0.923
39 0.921 0.946 0.882
40 0.878 0.914 0.866
41 0.922 0.948 0.898
42 0.895 0.935 0.91
43 0.925 0.943 0.904
44 0.923 0.944 0.938
45 0.962 0.954 0.916
46 0.913 0.948 0.92
47 0.931 0.955 0.931
48 0.945 0.95 0.915

download these results as csv

Task 1a Summary Results by Query

Simple Hit/Miss Counting [1]

MRR Method [2]

Task 1b, ThinkIT's dataset Results

Task 1b Overall Results
JSSLP1 TY1 TY2
Simple Count 0.941 0.456 0.899
MRR 0.912 0.437 0.853
Total Count 355 355 355

download these results as csv

Task 1b Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1b Simple/MRR data over the 106 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID TeamID Lowerbound Mean Upperbound Significance
JSSLP1 TY2 -0.0922 0.1604 0.4130 FALSE
JSSLP1 TY1 0.8465 1.0991 1.3516 TRUE
TY2 TY1 0.6861 0.9387 1.1913 TRUE

download these results as csv

2011QbshTask1bSimpleByGroup.friedman.tukeyKramerHSD.png

MRR Method:

TeamID TeamID Lowerbound Mean Upperbound Significance
JSSLP1 TY2 0.0002 0.2642 0.5281 TRUE
JSSLP1 TY1 0.8729 1.1368 1.4007 TRUE
TY2 TY1 0.6087 0.8726 1.1366 TRUE

download these results as csv

2011QbshTask1bMrrByGroup.friedman.tukeyKramerHSD.png

Task 1b Summary Results by Query Group

Simple Hit/Miss Count

JSSLP1 TY1 TY2
1 1 0.636 1
2 1 0.6 0.8
3 0 0 1
4 1 1 1
5 0 0 1
6 0.833 0 0.667
7 1 1 1
8 1 0.333 0.667
9 1 0 1
10 1 0 1
11 1 0 0.5
12 1 1 1
13 0.8 0.2 0.6
14 1 0 1
15 1 0 0.5
16 1 1 0.75
17 0 0 0
18 1 0.5 1
19 1 0 1
20 1 0 0.5
21 1 0 0.5
22 1 0.444 0.889
23 1 1 1
24 1 0 1
25 1 0.5 1
26 1 0.636 0.909
27 1 0.5 1
28 1 0.5 0.75
29 1 1 1
30 1 1 1
31 1 1 1
32 1 0 1
33 1 0 1
34 0.5 0.5 0.5
35 1 0 1
36 0.8 0 0.8
37 1 0 0
38 1 0.5 1
39 1 0.429 1
40 1 1 1
41 0.875 0.25 0.875
42 1 0.667 0.917
43 0.889 0.667 0.778
44 1 0.556 1
45 1 1 1
46 0.75 0.5 0.75
47 1 0.75 1
48 1 1 1
49 1 1 1
50 1 1 1
51 1 0.5 1
52 1 0.5 0.75
53 1 0.333 1
54 1 0.833 1
55 1 0 1
56 0.5 0.5 0.5
57 1 1 1
58 1 0 1
59 1 0 1
60 0 1 1
61 1 0 1
62 1 0.333 1
63 1 1 1
64 1 0 1
65 1 0 0.667
66 1 0.5 0.75
67 0.333 0 1
68 1 1 1
69 1 0 1
70 1 1 1
71 1 0 1
72 1 1 1
73 1 1 1
74 1 1 1
75 1 0.333 0.667
76 0.5 0 1
77 1 1 1
78 1 1 1
79 1 0.5 1
80 0.333 0.333 1
81 1 0.333 1
82 1 0 1
83 1 0 1
84 0.75 0.25 0.5
85 1 0.5 1
86 1 1 1
87 1 0 1
88 1 0 1
89 1 0.571 0.714
90 1 0 1
91 1 0 1
92 1 0.5 1
93 1 0.5 1
94 1 0 1
95 1 0 1
96 1 1 0
97 1 0 1
98 1 0 1
99 1 0.75 1
100 1 0.333 1
101 0 0 0
102 1 0 1
103 1 0.375 1
104 1 0 1
105 1 1 0
106 0.667 0 0.667

download these results as csv

MRR Method

JSSLP1 TY1 TY2
1 1 0.5 1
2 1 0.6 0.8
3 0 0 1
4 0.667 0.163 0.563
5 0 0 0.25
6 0.75 0 0.667
7 1 1 0.917
8 1 0.333 0.667
9 1 0 0.9
10 1 0 1
11 1 0 0.5
12 1 1 1
13 0.8 0.2 0.6
14 1 0 0.625
15 0.667 0 0.5
16 1 1 0.542
17 0 0 0
18 1 0.5 1
19 1 0 1
20 0.375 0 0.1
21 1 0 0.083
22 0.926 0.389 0.806
23 1 1 1
24 1 0 1
25 1 0.5 1
26 1 0.636 0.909
27 1 0.5 1
28 1 0.375 0.75
29 1 1 1
30 1 1 1
31 1 1 1
32 1 0 1
33 0.9 0 1
34 0.5 0.5 0.5
35 1 0 1
36 0.8 0 0.8
37 0.333 0 0
38 1 0.5 1
39 1 0.429 1
40 1 1 1
41 0.875 0.167 0.875
42 1 0.667 0.917
43 0.833 0.667 0.722
44 1 0.556 1
45 1 1 1
46 0.625 0.5 0.75
47 1 0.75 1
48 1 1 1
49 1 1 1
50 1 1 1
51 1 0.5 0.75
52 1 0.281 0.75
53 1 0.333 0.861
54 1 0.75 1
55 1 0 1
56 0.5 0.5 0.5
57 1 1 1
58 1 0 1
59 1 0 1
60 0 1 1
61 1 0 0.25
62 1 0.333 1
63 1 1 1
64 1 0 1
65 1 0 0.667
66 1 0.5 0.75
67 0.333 0 1
68 0.625 1 0.556
69 1 0 1
70 1 1 1
71 1 0 0.722
72 1 1 1
73 1 1 1
74 1 1 1
75 1 0.333 0.444
76 0.5 0 1
77 1 1 1
78 1 1 1
79 1 0.5 1
80 0.333 0.333 1
81 1 0.333 0.857
82 1 0 1
83 0.833 0 0.611
84 0.55 0.125 0.5
85 1 0.5 1
86 1 1 1
87 1 0 1
88 1 0 1
89 0.881 0.571 0.643
90 1 0 1
91 1 0 1
92 1 0.5 1
93 1 0.5 1
94 1 0 1
95 1 0 1
96 1 1 0
97 1 0 1
98 1 0 1
99 1 0.75 1
100 1 0.333 1
101 0 0 0
102 1 0 1
103 1 0.375 0.938
104 1 0 1
105 0.25 1 0
106 0.444 0 0.333

download these results as csv

Task 1b Summary Results by Query

Simple Hit/Miss Counting [3]

MRR Method [4]

Task 1c, IOACAS2's dataset Results

As argued by Jang, there are some abnormal in the IOACAS2 data set. Therefore, we just provide the raw result without any further analysis.


Task 1c Summary Results by Query Group

Simple Hit/Miss Count

JSSLP1 TY1 TY2
1 0 0 1
2 0 0 0
3 1 0 1
4 0 0 0
5 0.182 0.273 0.273
6 0 0 0
7 0 0.5 0.5
8 0 0 0
9 0 1 0
10 1 1 1
11 0 0 0.5
12 1 0 1
13 0 0.143 0.286
14 0.2 0.2 0.4
15 0 0 0
16 0 1 1
17 0 0 0
18 0 0 1
19 0 0 0.5
20 0 0.5 0
21 0 0 0.5
22 0 0 0
23 0 0 0
24 0 0 0
25 0 0 0
26 0 0 0.25
27 0 0 0
28 0 0 0
29 0.125 0.375 0.375
30 1 0 0
31 0 0 0
32 0 0 0
33 0 0.333 0.333
34 0 0 0
35 0 1 1
36 0 0.5 0
37 0.111 0 0.556
38 0 0.5 0.5
39 0 0 1
40 0.5 0.5 0.5
41 1 1 1
42 0 0 0
43 0 0 0
44 0.5 0.5 1
45 0 0 0
46 0 0 0
47 0 0 1
48 0 0 1
49 0 0 0
50 0.375 0.375 0.25
51 0 0.25 0.25
52 0 0.333 0.333
53 0 0 0.5
54 0.5 0 0.25
55 1 0 0
56 1 0.667 0.333
57 0.333 0 0.333
58 0 0 0
59 0 1 1
60 1 0 1
61 0 0 0.5
62 0 0 0
63 1 0 0
64 0.333 0.333 0.333
65 1 1 0
66 1 0 0
67 0.5 0 0
68 1 0.6 0.6
69 1 0 0.333
70 0.6 0 0.7
71 1 0 0
72 0.833 0.667 0.5
73 0 0 0
74 0.333 0 0
75 0.5 0 0
76 0 0 0
77 1 1 0
78 0.667 0.833 0.5
79 0 0 0
80 0.615 0.615 0.692
81 0.2 0 0
82 0 0 0
83 0.571 0.857 0.143
84 0 0 0
85 1 0 0
86 1 0 1
87 1 1 1
88 0.25 0.25 0.75
89 1 0 1
90 0 0 0
91 0.5 0.5 0.5
92 1 0.5 1
93 1 0.4 0.8
94 0 1 0
95 0 0 1
96 0.667 0.667 0.333
97 1 0 1
98 1 1 1
99 0 1 0
100 0 0 0.5
101 0 0 1
102 0.875 0.5 0.375
103 1 0 1
104 1 1 0
105 1 0 1
106 0 0 0
107 0.5 0 1
108 1 0 1
109 1 1 0.5
110 0 0 1
111 0 0 0
112 0 0 0
113 0 0 0
114 1 1 1
115 1 1 0
116 0 0.5 0
117 0 0 0
118 1 1 0
119 0 0.2 0.2
120 0 0 0
121 0 0 0
122 1 1 0
123 1 0 1
124 0 0 0
125 1 0.5 1
126 0 0 0
127 0 0 0
128 0 0 0
129 1 0 1
130 0 0 0
131 0 1 1
132 0 0 0.5
133 1 1 1
134 1 1 1
135 1 0 0
136 1 0 1
137 1 0 1
138 1 0 0.667
139 1 1 1
140 1 1 0
141 1 1 1
142 1 0 0.75
143 1 0 0
144 1 0 0
145 0.333 0.333 0.333
146 0 0 0
147 1 0 1
148 1 0.5 1
149 1 1 1
150 1 0.5 1
151 1 0 1
152 0.667 0.333 0.333
153 1 0 0
154 1 0 1
155 0 0 1
156 1 1 0
157 1 1 1
158 0 0 0
159 0 0 0
160 0 0 0
161 0 1 0
162 0 1 0
163 0 0 0
164 1 0 1
165 1 0 0
166 0 0 0
167 1 1 1
168 1 1 1
169 1 1 1
170 1 1 1
171 0 0 0
172 1 1 1
173 1 1 1
174 1 1 0
175 1 0 1
176 1 0 1
177 0 0 0
178 1 1 0.5
179 1 1 1
180 1 0 0
181 1 1 1
182 1 0 0
183 1 0 1
184 0 0 1
185 0 1 1
186 1 0 0.667
187 1 0 1
188 0 1 0
189 0 0 1
190 1 0.5 1
191 1 1 0
192 0 0 1

download these results as csv

MRR Method

JSSLP1 TY1 TY2
1 0 0 1
2 0 0 0
3 1 0 1
4 0 0 0
5 0.025 0.132 0.133
6 0 0 0
7 0 0.5 0.083
8 0 0 0
9 0 1 0
10 1 1 1
11 0 0 0.3
12 1 0 1
13 0 0.143 0.163
14 0.04 0.022 0.073
15 0 0 0
16 0 1 0.5
17 0 0 0
18 0 0 0.2
19 0 0 0.071
20 0 0.092 0
21 0 0 0.063
22 0 0 0
23 0 0 0
24 0 0 0
25 0 0 0
26 0 0 0.042
27 0 0 0
28 0 0 0
29 0.042 0.375 0.292
30 0.333 0 0
31 0 0 0
32 0 0 0
33 0 0.167 0.083
34 0 0 0
35 0 0.1 1
36 0 0.5 0
37 0.037 0 0.347
38 0 0.5 0.167
39 0 0 0.5
40 0.5 0.5 0.25
41 1 0.1 0.5
42 0 0 0
43 0 0 0
44 0.5 0.5 0.75
45 0 0 0
46 0 0 0
47 0 0 0.25
48 0 0 0.5
49 0 0 0
50 0.156 0.153 0.104
51 0 0.025 0.25
52 0 0.333 0.111
53 0 0 0.5
54 0.5 0 0.031
55 1 0 0
56 1 0.25 0.167
57 0.333 0 0.042
58 0 0 0
59 0 1 1
60 1 0 1
61 0 0 0.25
62 0 0 0
63 1 0 0
64 0.333 0.333 0.235
65 0.25 0.5 0
66 1 0 0
67 0.167 0 0
68 0.808 0.511 0.448
69 1 0 0.167
70 0.513 0 0.7
71 1 0 0
72 0.833 0.542 0.357
73 0 0 0
74 0.063 0 0
75 0.125 0 0
76 0 0 0
77 1 0.333 0
78 0.431 0.597 0.274
79 0 0 0
80 0.492 0.547 0.564
81 0.1 0 0
82 0 0 0
83 0.214 0.619 0.048
84 0 0 0
85 1 0 0
86 1 0 0.813
87 0.143 1 0.333
88 0.25 0.25 0.625
89 1 0 1
90 0 0 0
91 0.5 0.5 0.5
92 1 0.5 1
93 0.867 0.4 0.65
94 0 0.143 0
95 0 0 0.143
96 0.667 0.667 0.333
97 1 0 0.6
98 1 1 0.583
99 0 1 0
100 0 0 0.167
101 0 0 0.1
102 0.688 0.5 0.375
103 1 0 1
104 0.143 1 0
105 1 0 1
106 0 0 0
107 0.063 0 1
108 1 0 0.333
109 0.75 1 0.5
110 0 0 0.5
111 0 0 0
112 0 0 0
113 0 0 0
114 1 1 1
115 1 0.75 0
116 0 0.5 0
117 0 0 0
118 1 0.333 0
119 0 0.1 0.2
120 0 0 0
121 0 0 0
122 0.143 1 0
123 1 0 0.333
124 0 0 0
125 1 0.063 1
126 0 0 0
127 0 0 0
128 0 0 0
129 0.111 0 1
130 0 0 0
131 0 1 1
132 0 0 0.167
133 0.5 1 1
134 1 1 1
135 0.5 0 0
136 1 0 1
137 0.143 0 0.143
138 0.833 0 0.278
139 1 1 1
140 1 0.5 0
141 1 1 1
142 0.708 0 0.165
143 1 0 0
144 1 0 0
145 0.333 0.333 0.333
146 0 0 0
147 0.25 0 0.5
148 1 0.5 1
149 1 1 1
150 0.571 0.5 0.5
151 1 0 0.5
152 0.444 0.111 0.333
153 1 0 0
154 0.25 0 0.5
155 0 0 0.2
156 0.2 1 0
157 1 1 1
158 0 0 0
159 0 0 0
160 0 0 0
161 0 0.25 0
162 0 0.333 0
163 0 0 0
164 0.333 0 1
165 1 0 0
166 0 0 0
167 0.5 1 1
168 1 1 0.5
169 1 1 1
170 1 1 0.167
171 0 0 0
172 1 1 1
173 1 1 1
174 1 0.111 0
175 1 0 0.5
176 1 0 0.5
177 0 0 0
178 0.625 1 0.063
179 1 1 1
180 0.5 0 0
181 1 1 1
182 1 0 0
183 1 0 1
184 0 0 1
185 0 0.125 0.333
186 1 0 0.5
187 1 0 0.333
188 0 0.125 0
189 0 0 1
190 1 0.5 0.417
191 1 0.5 0
192 0 0 1

download these results as csv

Task 1c Summary Results by Query

Simple Hit/Miss Counting [5]

MRR Method [6]

Task 2 Results

Task 2 Overall Results
JSSLP1 TY1 TY2
Multi Count 9.275 8.744 8.744
Total Count 2040 2040 2040

download these results as csv

Task 2 Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 2 data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

TeamID TeamID Lowerbound Mean Upperbound Significance
JSSLP1 TY2 0.2107 0.6250 1.0393 TRUE
JSSLP1 TY1 0.2107 0.6250 1.0393 TRUE
TY2 TY1 -0.4143 0.0000 0.4143 FALSE

download these results as csv

2011QbshTask2Group.friedman.tukeyKramerHSD.png


Task 2 Summary Results by Query Group
JSSLP1 TY1 TY2
1 5.889 5.333 5.333
2 2.000 2.250 2.250
3 4.000 0.000 0.000
4 3.200 4.400 4.400
5 7.000 6.800 6.800
6 3.000 5.000 5.000
7 2.800 6.600 6.600
8 8.600 6.800 6.800
9 3.500 2.000 2.000
10 5.889 4.556 4.556
11 9.349 8.791 8.791
12 9.222 9.542 9.542
13 9.145 9.237 9.237
14 9.625 8.722 8.722
15 8.071 5.214 5.214
16 9.757 8.932 8.932
17 9.167 9.181 9.181
18 9.789 9.026 9.026
19 9.478 9.333 9.333
20 9.659 9.341 9.341
21 8.735 8.971 8.971
22 9.663 9.398 9.398
23 9.581 8.516 8.516
24 8.898 9.136 9.136
25 8.898 8.119 8.119
26 9.095 7.619 7.619
27 9.267 8.156 8.156
28 9.152 9.043 9.043
29 9.956 9.426 9.426
30 9.418 9.687 9.687
31 9.800 9.650 9.650
32 9.484 9.719 9.719
33 8.980 7.824 7.824
34 9.721 9.456 9.456
35 9.710 8.971 8.971
36 7.167 7.833 7.833
37 8.893 7.250 7.250
38 9.741 8.593 8.593
39 9.662 7.574 7.574
40 9.481 8.815 8.815
41 9.136 9.322 9.322
42 9.071 7.119 7.119
43 9.483 8.483 8.483
44 8.778 9.361 9.361
45 9.574 7.298 7.298
46 9.368 8.368 8.368
47 8.769 8.231 8.231
48 8.636 8.364 8.364

download these results as csv

Task 2 Summary Results by Query

JSSLP1 TY1 TY2
1 2 6 6
2 5 7 7
3 7 6 6
4 8 8 8
5 3 8 8
6 5 7 7
7 7 7 7
8 6 8 8
9 8 8 8
10 7 7 7
11 6 9 9
12 7 8 8
13 7 7 7
14 8 8 8
15 4 5 5
16 0 1 1
17 7 7 7
18 8 7 7
19 4 0 0
20 9 8 8
21 6 10 10
22 10 10 10
23 10 10 10
24 10 9 9
25 2 10 10
26 10 9 9
27 10 10 10
28 10 10 10
29 9 10 10
30 10 10 10
31 10 10 10
32 10 10 10
33 10 10 10
34 10 10 10
35 10 10 10
36 10 10 10
37 7 8 8
38 5 10 10
39 10 10 10
40 4 10 10
41 10 10 10
42 7 10 10
43 10 10 10
44 10 10 10
45 10 10 10
46 9 9 9
47 10 10 10
48 10 10 10
49 10 8 8
50 10 7 7
51 10 10 10
52 10 10 10
53 10 10 10
54 10 10 10
55 10 7 7
56 10 10 10
57 10 10 10
58 10 10 10
59 10 9 9
60 10 10 10
61 10 10 10
62 10 10 10
63 10 6 6
64 10 10 10
65 10 10 10
66 10 10 10
67 10 9 9
68 10 9 9
69 10 10 10
70 10 10 10
71 10 10 10
72 10 8 8
73 10 10 10
74 10 8 8
75 10 10 10
76 10 10 10
77 10 10 10
78 10 9 9
79 10 8 8
80 10 10 10
81 5 10 10
82 10 10 10
83 10 9 9
84 10 9 9
85 10 9 9
86 10 10 10
87 5 10 10
88 10 10 10
89 10 8 8
90 10 10 10
91 7 10 10
92 10 9 9
93 10 10 10
94 10 10 10
95 10 6 6
96 10 10 10
97 10 10 10
98 6 4 4
99 10 10 10
100 10 10 10
101 10 10 10
102 10 8 8
103 10 10 10
104 10 10 10
105 10 10 10
106 10 10 10
107 10 10 10
108 10 10 10
109 10 10 10
110 10 10 10
111 10 10 10
112 10 9 9
113 10 10 10
114 10 9 9
115 10 10 10
116 10 10 10
117 10 10 10
118 10 10 10
119 10 10 10
120 10 10 10
121 10 10 10
122 10 10 10
123 10 10 10
124 10 10 10
125 10 10 10
126 10 10 10
127 10 10 10
128 10 6 6
129 10 10 10
130 10 10 10
131 10 10 10
132 10 10 10
133 10 10 10
134 10 10 10
135 10 10 10
136 10 10 10
137 10 8 8
138 10 10 10
139 10 10 10
140 10 10 10
141 8 9 9
142 10 10 10
143 10 8 8
144 10 8 8
145 10 9 9
146 10 10 10
147 10 10 10
148 8 3 3
149 10 10 10
150 10 10 10
151 10 9 9
152 10 10 10
153 10 10 10
154 10 10 10
155 10 10 10
156 10 10 10
157 10 9 9
158 9 9 9
159 10 10 10
160 9 10 10
161 10 4 4
162 10 7 7
163 7 10 10
164 10 10 10
165 10 7 7
166 10 9 9
167 10 10 10
168 10 10 10
169 10 10 10
170 10 9 9
171 10 10 10
172 10 10 10
173 10 10 10
174 10 10 10
175 10 10 10
176 10 10 10
177 10 10 10
178 10 10 10
179 10 10 10
180 10 10 10
181 10 10 10
182 10 10 10
183 10 10 10
184 10 10 10
185 10 10 10
186 10 8 8
187 10 10 10
188 10 3 3
189 8 6 6
190 10 1 1
191 9 10 10
192 10 10 10
193 7 1 1
194 10 10 10
195 5 5 5
196 8 6 6
197 10 7 7
198 10 10 10
199 10 9 9
200 10 10 10
201 10 9 9
202 10 10 10
203 10 10 10
204 10 5 5
205 10 10 10
206 10 9 9
207 10 10 10
208 10 9 9
209 10 10 10
210 10 10 10
211 10 10 10
212 10 9 9
213 10 10 10
214 10 10 10
215 10 10 10
216 10 10 10
217 10 10 10
218 10 10 10
219 10 10 10
220 10 10 10
221 10 10 10
222 10 10 10
223 10 10 10
224 10 10 10
225 10 10 10
226 10 10 10
227 10 10 10
228 10 10 10
229 10 10 10
230 10 8 8
231 10 10 10
232 10 10 10
233 10 9 9
234 10 10 10
235 10 10 10
236 10 9 9
237 10 10 10
238 10 10 10
239 10 10 10
240 10 10 10
241 10 10 10
242 10 9 9
243 10 9 9
244 10 10 10
245 10 9 9
246 10 10 10
247 10 10 10
248 10 10 10
249 10 10 10
250 10 10 10
251 10 10 10
252 10 10 10
253 10 10 10
254 10 10 10
255 9 10 10
256 7 10 10
257 10 10 10
258 10 10 10
259 10 10 10
260 10 10 10
261 10 10 10
262 10 10 10
263 10 10 10
264 10 9 9
265 10 10 10
266 6 10 10
267 10 10 10
268 10 8 8
269 10 10 10
270 4 5 5
271 10 9 9
272 10 10 10
273 10 10 10
274 10 10 10
275 10 10 10
276 10 4 4
277 10 10 10
278 10 10 10
279 10 10 10
280 10 10 10
281 10 10 10
282 10 10 10
283 10 10 10
284 10 10 10
285 10 9 9
286 10 10 10
287 10 10 10
288 10 10 10
289 10 10 10
290 10 10 10
291 9 6 6
292 10 10 10
293 9 7 7
294 10 8 8
295 10 8 8
296 10 10 10
297 9 9 9
298 10 10 10
299 10 8 8
300 10 10 10
301 10 9 9
302 9 5 5
303 8 8 8
304 10 5 5
305 10 10 10
306 10 10 10
307 10 9 9
308 10 5 5
309 10 10 10
310 9 8 8
311 10 10 10
312 10 10 10
313 10 10 10
314 10 9 9
315 10 10 10
316 0 0 0
317 10 10 10
318 10 10 10
319 10 8 8
320 10 10 10
321 10 10 10
322 2 5 5
323 10 10 10
324 10 10 10
325 10 9 9
326 10 9 9
327 8 5 5
328 10 10 10
329 10 9 9
330 10 10 10
331 10 10 10
332 10 10 10
333 10 10 10
334 10 10 10
335 10 10 10
336 10 10 10
337 10 10 10
338 10 10 10
339 10 10 10
340 10 9 9
341 10 10 10
342 10 10 10
343 10 10 10
344 10 10 10
345 10 10 10
346 10 10 10
347 10 10 10
348 10 10 10
349 4 8 8
350 10 10 10
351 9 9 9
352 9 10 10
353 9 9 9
354 6 10 10
355 10 9 9
356 10 9 9
357 8 8 8
358 10 3 3
359 10 10 10
360 10 10 10
361 10 10 10
362 4 9 9
363 10 9 9
364 9 10 10
365 4 8 8
366 10 8 8
367 9 10 10
368 10 10 10
369 10 10 10
370 10 10 10
371 10 10 10
372 10 10 10
373 10 10 10
374 10 10 10
375 10 10 10
376 10 10 10
377 10 10 10
378 10 10 10
379 10 7 7
380 10 10 10
381 10 10 10
382 10 10 10
383 10 10 10
384 9 10 10
385 10 10 10
386 10 10 10
387 10 10 10
388 10 10 10
389 10 10 10
390 10 10 10
391 10 10 10
392 10 10 10
393 10 10 10
394 9 7 7
395 10 2 2
396 10 10 10
397 10 10 10
398 10 10 10
399 10 10 10
400 10 8 8
401 10 10 10
402 10 10 10
403 10 10 10
404 10 10 10
405 10 10 10
406 10 10 10
407 7 9 9
408 10 9 9
409 10 10 10
410 10 10 10
411 10 10 10
412 10 10 10
413 10 10 10
414 10 10 10
415 10 10 10
416 10 10 10
417 7 10 10
418 10 10 10
419 10 10 10
420 10 10 10
421 10 10 10
422 10 10 10
423 10 10 10
424 10 10 10
425 10 3 3
426 10 8 8
427 10 10 10
428 3 4 4
429 10 7 7
430 3 9 9
431 10 10 10
432 7 10 10
433 6 7 7
434 10 7 7
435 9 9 9
436 10 10 10
437 10 9 9
438 10 8 8
439 10 9 9
440 10 10 10
441 10 0 0
442 1 3 3
443 10 10 10
444 10 10 10
445 10 7 7
446 9 1 1
447 7 0 0
448 9 9 9
449 4 8 8
450 10 2 2
451 10 2 2
452 10 2 2
453 9 9 9
454 10 8 8
455 10 2 2
456 8 6 6
457 9 3 3
458 8 9 9
459 3 3 3
460 10 10 10
461 10 10 10
462 10 8 8
463 4 9 9
464 10 10 10
465 10 4 4
466 10 10 10
467 10 10 10
468 10 10 10
469 9 8 8
470 9 10 10
471 10 10 10
472 10 10 10
473 10 9 9
474 10 9 9
475 10 10 10
476 10 10 10
477 8 10 10
478 10 9 9
479 10 10 10
480 10 10 10
481 10 8 8
482 10 7 7
483 10 10 10
484 10 5 5
485 10 10 10
486 10 10 10
487 9 9 9
488 10 10 10
489 10 9 9
490 10 8 8
491 10 10 10
492 10 10 10
493 10 10 10
494 10 10 10
495 10 10 10
496 10 9 9
497 10 10 10
498 10 9 9
499 8 9 9
500 10 10 10
501 10 10 10
502 10 10 10
503 10 10 10
504 10 10 10
505 10 10 10
506 10 9 9
507 10 10 10
508 10 10 10
509 10 10 10
510 10 10 10
511 10 10 10
512 10 10 10
513 10 10 10
514 10 6 6
515 10 10 10
516 10 10 10
517 10 10 10
518 10 10 10
519 10 10 10
520 7 10 10
521 10 10 10
522 1 5 5
523 10 10 10
524 10 10 10
525 7 10 10
526 10 10 10
527 5 7 7
528 10 0 0
529 9 9 9
530 10 3 3
531 3 10 10
532 5 9 9
533 10 9 9
534 3 10 10
535 10 10 10
536 10 10 10
537 9 7 7
538 8 10 10
539 10 10 10
540 8 10 10
541 10 10 10
542 9 7 7
543 10 10 10
544 10 10 10
545 10 8 8
546 10 10 10
547 10 10 10
548 10 8 8
549 10 8 8
550 10 10 10
551 10 10 10
552 10 10 10
553 10 7 7
554 10 10 10
555 10 10 10
556 10 10 10
557 10 10 10
558 10 10 10
559 10 10 10
560 10 10 10
561 10 10 10
562 10 10 10
563 10 9 9
564 10 9 9
565 10 10 10
566 10 10 10
567 10 10 10
568 10 10 10
569 10 6 6
570 10 10 10
571 10 10 10
572 10 10 10
573 10 10 10
574 10 10 10
575 10 10 10
576 10 10 10
577 10 4 4
578 10 10 10
579 10 9 9
580 10 10 10
581 10 10 10
582 10 10 10
583 10 10 10
584 10 10 10
585 10 10 10
586 10 10 10
587 10 10 10
588 10 10 10
589 10 10 10
590 10 10 10
591 10 10 10
592 10 10 10
593 10 10 10
594 10 10 10
595 10 10 10
596 10 8 8
597 10 10 10
598 10 10 10
599 10 10 10
600 10 10 10
601 10 10 10
602 10 10 10
603 10 10 10
604 10 10 10
605 10 10 10
606 10 8 8
607 10 10 10
608 10 9 9
609 10 10 10
610 10 10 10
611 10 10 10
612 10 10 10
613 10 10 10
614 10 10 10
615 10 10 10
616 10 9 9
617 10 10 10
618 10 10 10
619 10 10 10
620 10 5 5
621 10 10 10
622 10 10 10
623 10 10 10
624 10 5 5
625 10 10 10
626 10 10 10
627 10 9 9
628 7 5 5
629 10 10 10
630 10 9 9
631 10 10 10
632 10 10 10
633 10 10 10
634 10 8 8
635 10 10 10
636 10 9 9
637 10 7 7
638 10 9 9
639 10 10 10
640 10 8 8
641 10 2 2
642 10 10 10
643 10 10 10
644 7 10 10
645 10 7 7
646 9 9 9
647 10 10 10
648 10 9 9
649 10 10 10
650 10 8 8
651 10 10 10
652 10 9 9
653 10 10 10
654 10 10 10
655 10 10 10
656 10 10 10
657 10 9 9
658 10 10 10
659 10 10 10
660 10 10 10
661 10 10 10
662 10 5 5
663 10 10 10
664 10 10 10
665 3 10 10
666 9 8 8
667 10 10 10
668 2 8 8
669 0 8 8
670 10 10 10
671 10 2 2
672 10 6 6
673 7 1 1
674 10 10 10
675 8 8 8
676 10 10 10
677 10 9 9
678 0 1 1
679 9 6 6
680 10 1 1
681 0 10 10
682 10 10 10
683 10 10 10
684 8 5 5
685 9 10 10
686 10 10 10
687 6 0 0
688 10 10 10
689 9 9 9
690 10 10 10
691 6 6 6
692 10 10 10
693 10 7 7
694 10 10 10
695 10 10 10
696 10 3 3
697 10 10 10
698 8 2 2
699 10 10 10
700 2 9 9
701 9 10 10
702 3 8 8
703 9 3 3
704 10 6 6
705 5 8 8
706 10 4 4
707 6 4 4
708 10 2 2
709 10 1 1
710 10 3 3
711 9 8 8
712 7 8 8
713 10 6 6
714 10 7 7
715 10 9 9
716 10 6 6
717 10 1 1
718 1 10 10
719 10 3 3
720 10 0 0
721 5 10 10
722 10 7 7
723 10 10 10
724 10 9 9
725 10 8 8
726 10 10 10
727 10 10 10
728 10 10 10
729 10 10 10
730 10 10 10
731 10 10 10
732 10 9 9
733 10 10 10
734 10 10 10
735 10 8 8
736 10 10 10
737 10 10 10
738 10 10 10
739 10 10 10
740 10 10 10
741 10 10 10
742 10 9 9
743 4 8 8
744 10 10 10
745 10 9 9
746 10 5 5
747 0 8 8
748 10 10 10
749 10 10 10
750 10 8 8
751 10 2 2
752 1 7 7
753 8 4 4
754 0 1 1
755 10 10 10
756 10 4 4
757 10 1 1
758 10 10 10
759 9 1 1
760 10 10 10
761 9 9 9
762 8 1 1
763 3 9 9
764 10 10 10
765 5 4 4
766 8 9 9
767 10 10 10
768 10 9 9
769 10 1 1
770 10 9 9
771 10 10 10
772 10 7 7
773 10 4 4
774 10 10 10
775 10 10 10
776 6 6 6
777 10 10 10
778 10 10 10
779 10 6 6
780 7 4 4
781 10 10 10
782 10 10 10
783 10 8 8
784 10 10 10
785 10 10 10
786 6 4 4
787 10 5 5
788 10 10 10
789 10 10 10
790 10 10 10
791 6 10 10
792 10 10 10
793 10 10 10
794 10 10 10
795 10 8 8
796 10 10 10
797 10 10 10
798 0 0 0
799 10 10 10
800 10 8 8
801 10 10 10
802 10 8 8
803 10 10 10
804 10 10 10
805 10 10 10
806 10 8 8
807 10 10 10
808 10 10 10
809 10 9 9
810 10 10 10
811 10 7 7
812 9 10 10
813 10 10 10
814 10 10 10
815 10 10 10
816 10 10 10
817 10 10 10
818 10 10 10
819 10 10 10
820 10 10 10
821 10 10 10
822 8 9 9
823 9 10 10
824 10 10 10
825 10 10 10
826 10 10 10
827 10 10 10
828 10 10 10
829 10 7 7
830 10 7 7
831 10 10 10
832 10 6 6
833 10 10 10
834 10 10 10
835 10 10 10
836 10 10 10
837 10 10 10
838 10 3 3
839 10 10 10
840 10 10 10
841 10 10 10
842 10 10 10
843 10 10 10
844 10 10 10
845 10 10 10
846 10 10 10
847 10 10 10
848 10 10 10
849 10 7 7
850 10 10 10
851 10 9 9
852 10 10 10
853 10 10 10
854 10 8 8
855 10 10 10
856 10 10 10
857 10 10 10
858 10 10 10
859 10 10 10
860 10 10 10
861 10 10 10
862 10 10 10
863 10 8 8
864 10 9 9
865 10 10 10
866 10 10 10
867 10 10 10
868 10 10 10
869 10 8 8
870 10 10 10
871 10 9 9
872 10 10 10
873 10 10 10
874 10 9 9
875 10 10 10
876 10 10 10
877 10 10 10
878 10 10 10
879 10 10 10
880 10 10 10
881 10 10 10
882 10 10 10
883 10 10 10
884 10 10 10
885 10 10 10
886 10 10 10
887 10 10 10
888 10 10 10
889 10 10 10
890 9 9 9
891 10 10 10
892 10 10 10
893 10 8 8
894 10 10 10
895 8 10 10
896 10 10 10
897 10 10 10
898 10 10 10
899 10 10 10
900 10 10 10
901 8 10 10
902 10 10 10
903 10 10 10
904 10 4 4
905 10 10 10
906 2 10 10
907 10 10 10
908 10 10 10
909 6 4 4
910 10 10 10
911 8 8 8
912 10 10 10
913 10 9 9
914 10 10 10
915 9 3 3
916 10 10 10
917 10 10 10
918 10 10 10
919 10 10 10
920 10 10 10
921 10 10 10
922 10 10 10
923 10 8 8
924 10 7 7
925 10 10 10
926 10 9 9
927 10 9 9
928 7 1 1
929 10 10 10
930 10 10 10
931 10 10 10
932 10 9 9
933 9 8 8
934 10 10 10
935 10 10 10
936 10 10 10
937 10 10 10
938 10 10 10
939 10 8 8
940 10 10 10
941 10 5 5
942 10 10 10
943 10 10 10
944 10 10 10
945 10 9 9
946 10 10 10
947 10 10 10
948 10 10 10
949 10 10 10
950 10 10 10
951 10 8 8
952 10 10 10
953 10 10 10
954 10 10 10
955 10 10 10
956 10 10 10
957 10 10 10
958 10 5 5
959 9 8 8
960 10 10 10
961 9 10 10
962 8 9 9
963 10 10 10
964 0 10 10
965 10 10 10
966 10 10 10
967 10 10 10
968 10 10 10
969 10 10 10
970 10 10 10
971 10 10 10
972 10 9 9
973 10 10 10
974 10 10 10
975 10 6 6
976 10 10 10
977 10 10 10
978 8 9 9
979 10 10 10
980 10 0 0
981 10 10 10
982 10 9 9
983 10 4 4
984 10 6 6
985 10 8 8
986 10 10 10
987 10 10 10
988 2 0 0
989 8 1 1
990 10 8 8
991 10 10 10
992 10 7 7
993 10 10 10
994 10 10 10
995 10 10 10
996 4 9 9
997 10 1 1
998 10 10 10
999 10 10 10
1000 10 10 10
1001 10 4 4
1002 10 9 9
1003 10 10 10
1004 10 10 10
1005 10 7 7
1006 9 9 9
1007 10 10 10
1008 6 5 5
1009 8 7 7
1010 10 1 1
1011 10 10 10
1012 10 10 10
1013 10 10 10
1014 10 8 8
1015 10 9 9
1016 10 9 9
1017 10 6 6
1018 7 8 8
1019 10 9 9
1020 10 10 10
1021 9 8 8
1022 10 10 10
1023 10 9 9
1024 7 10 10
1025 10 9 9
1026 9 10 10
1027 10 10 10
1028 10 9 9
1029 10 9 9
1030 2 10 10
1031 10 10 10
1032 10 10 10
1033 10 10 10
1034 10 7 7
1035 10 10 10
1036 10 10 10
1037 10 8 8
1038 9 9 9
1039 10 10 10
1040 10 8 8
1041 10 10 10
1042 10 10 10
1043 10 9 9
1044 10 10 10
1045 10 10 10
1046 10 10 10
1047 10 7 7
1048 10 10 10
1049 10 10 10
1050 10 10 10
1051 10 10 10
1052 10 10 10
1053 10 10 10
1054 10 10 10
1055 10 10 10
1056 10 10 10
1057 10 10 10
1058 6 9 9
1059 10 5 5
1060 10 10 10
1061 10 8 8
1062 10 9 9
1063 10 10 10
1064 10 10 10
1065 9 4 4
1066 10 1 1
1067 10 10 10
1068 10 10 10
1069 10 8 8
1070 10 10 10
1071 10 10 10
1072 10 10 10
1073 6 3 3
1074 10 10 10
1075 10 10 10
1076 10 8 8
1077 9 9 9
1078 10 10 10
1079 10 7 7
1080 8 10 10
1081 10 10 10
1082 10 10 10
1083 10 10 10
1084 10 10 10
1085 5 10 10
1086 10 10 10
1087 10 10 10
1088 10 10 10
1089 10 10 10
1090 2 10 10
1091 10 8 8
1092 8 10 10
1093 6 10 10
1094 10 10 10
1095 6 10 10
1096 6 10 10
1097 10 10 10
1098 9 10 10
1099 10 8 8
1100 10 10 10
1101 10 10 10
1102 10 10 10
1103 10 10 10
1104 10 10 10
1105 10 10 10
1106 10 10 10
1107 10 10 10
1108 10 10 10
1109 10 10 10
1110 10 10 10
1111 10 10 10
1112 10 10 10
1113 10 10 10
1114 10 10 10
1115 10 10 10
1116 10 10 10
1117 10 10 10
1118 10 10 10
1119 10 10 10
1120 10 10 10
1121 10 10 10
1122 10 8 8
1123 10 10 10
1124 10 10 10
1125 10 8 8
1126 10 10 10
1127 10 10 10
1128 10 10 10
1129 10 7 7
1130 9 10 10
1131 5 9 9
1132 10 9 9
1133 10 10 10
1134 10 10 10
1135 10 8 8
1136 10 10 10
1137 8 9 9
1138 10 9 9
1139 10 9 9
1140 10 10 10
1141 10 5 5
1142 10 7 7
1143 10 5 5
1144 10 10 10
1145 10 8 8
1146 9 9 9
1147 10 10 10
1148 8 10 10
1149 10 6 6
1150 5 8 8
1151 10 8 8
1152 10 0 0
1153 8 7 7
1154 5 9 9
1155 10 10 10
1156 10 10 10
1157 10 0 0
1158 10 10 10
1159 10 10 10
1160 1 8 8
1161 10 10 10
1162 10 10 10
1163 10 10 10
1164 9 10 10
1165 10 10 10
1166 10 5 5
1167 10 10 10
1168 10 10 10
1169 10 10 10
1170 10 10 10
1171 10 10 10
1172 10 10 10
1173 10 10 10
1174 10 10 10
1175 10 10 10
1176 10 10 10
1177 10 10 10
1178 10 10 10
1179 10 10 10
1180 10 10 10
1181 10 10 10
1182 10 4 4
1183 10 8 8
1184 5 10 10
1185 5 6 6
1186 10 4 4
1187 10 10 10
1188 10 10 10
1189 10 10 10
1190 10 8 8
1191 10 1 1
1192 4 9 9
1193 10 10 10
1194 10 10 10
1195 10 10 10
1196 10 10 10
1197 10 10 10
1198 10 9 9
1199 10 10 10
1200 10 4 4
1201 3 1 1
1202 10 10 10
1203 10 10 10
1204 10 10 10
1205 1 0 0
1206 0 0 0
1207 3 10 10
1208 1 0 0
1209 8 8 8
1210 0 3 3
1211 1 1 1
1212 10 7 7
1213 7 9 9
1214 1 2 2
1215 0 0 0
1216 4 1 1
1217 0 2 2
1218 4 1 1
1219 7 3 3
1220 10 5 5
1221 10 10 10
1222 0 1 1
1223 8 10 10
1224 4 10 10
1225 10 9 9
1226 0 0 0
1227 0 0 0
1228 7 0 0
1229 10 10 10
1230 10 8 8
1231 10 10 10
1232 10 9 9
1233 10 10 10
1234 10 10 10
1235 10 10 10
1236 10 10 10
1237 10 10 10
1238 5 10 10
1239 10 10 10
1240 10 10 10
1241 10 10 10
1242 10 10 10
1243 10 10 10
1244 10 10 10
1245 10 9 9
1246 10 10 10
1247 10 10 10
1248 10 8 8
1249 10 10 10
1250 10 10 10
1251 0 1 1
1252 5 9 9
1253 5 3 3
1254 10 7 7
1255 0 1 1
1256 0 0 0
1257 7 1 1
1258 3 0 0
1259 5 4 4
1260 1 0 0
1261 10 1 1
1262 2 1 1
1263 10 5 5
1264 0 0 0
1265 10 10 10
1266 5 2 2
1267 3 0 0
1268 6 5 5
1269 9 4 4
1270 7 2 2
1271 7 10 10
1272 10 10 10
1273 10 8 8
1274 7 9 9
1275 9 10 10
1276 10 2 2
1277 10 9 9
1278 10 10 10
1279 10 10 10
1280 10 10 10
1281 10 10 10
1282 10 10 10
1283 10 10 10
1284 10 9 9
1285 10 8 8
1286 10 10 10
1287 10 9 9
1288 10 10 10
1289 10 10 10
1290 10 10 10
1291 10 10 10
1292 10 6 6
1293 10 10 10
1294 10 10 10
1295 9 10 10
1296 10 9 9
1297 10 8 8
1298 10 10 10
1299 10 10 10
1300 10 9 9
1301 10 10 10
1302 10 10 10
1303 10 10 10
1304 10 10 10
1305 10 6 6
1306 10 10 10
1307 10 10 10
1308 9 2 2
1309 10 10 10
1310 10 10 10
1311 10 3 3
1312 10 10 10
1313 9 9 9
1314 10 2 2
1315 10 10 10
1316 10 9 9
1317 10 10 10
1318 10 8 8
1319 10 8 8
1320 10 10 10
1321 10 10 10
1322 10 10 10
1323 10 10 10
1324 10 10 10
1325 10 8 8
1326 10 10 10
1327 10 10 10
1328 10 10 10
1329 10 10 10
1330 10 10 10
1331 10 10 10
1332 10 10 10
1333 10 10 10
1334 10 6 6
1335 10 10 10
1336 10 9 9
1337 10 5 5
1338 10 10 10
1339 10 10 10
1340 10 9 9
1341 10 10 10
1342 6 4 4
1343 10 8 8
1344 10 9 9
1345 10 8 8
1346 10 9 9
1347 10 10 10
1348 10 10 10
1349 10 10 10
1350 10 10 10
1351 10 10 10
1352 10 10 10
1353 10 4 4
1354 10 10 10
1355 10 6 6
1356 10 10 10
1357 10 10 10
1358 10 10 10
1359 10 8 8
1360 9 8 8
1361 10 10 10
1362 10 10 10
1363 10 10 10
1364 10 10 10
1365 10 10 10
1366 10 10 10
1367 4 4 4
1368 10 9 9
1369 10 10 10
1370 10 10 10
1371 10 9 9
1372 10 10 10
1373 10 10 10
1374 10 10 10
1375 10 7 7
1376 10 10 10
1377 10 9 9
1378 9 9 9
1379 10 10 10
1380 10 2 2
1381 5 7 7
1382 10 9 9
1383 9 9 9
1384 6 2 2
1385 10 1 1
1386 10 3 3
1387 10 10 10
1388 3 5 5
1389 5 2 2
1390 10 8 8
1391 10 9 9
1392 10 3 3
1393 10 9 9
1394 2 10 10
1395 10 9 9
1396 10 10 10
1397 10 9 9
1398 10 9 9
1399 10 9 9
1400 10 9 9
1401 10 9 9
1402 9 9 9
1403 10 2 2
1404 10 10 10
1405 10 1 1
1406 10 8 8
1407 10 10 10
1408 10 10 10
1409 10 9 9
1410 10 10 10
1411 10 9 9
1412 10 10 10
1413 10 10 10
1414 10 10 10
1415 10 1 1
1416 10 10 10
1417 10 10 10
1418 10 8 8
1419 10 10 10
1420 10 10 10
1421 1 6 6
1422 10 10 10
1423 10 10 10
1424 10 10 10
1425 10 10 10
1426 10 10 10
1427 10 10 10
1428 10 10 10
1429 10 10 10
1430 10 10 10
1431 10 10 10
1432 10 10 10
1433 10 10 10
1434 10 10 10
1435 10 10 10
1436 10 10 10
1437 10 10 10
1438 10 9 9
1439 10 10 10
1440 10 10 10
1441 10 10 10
1442 10 10 10
1443 9 4 4
1444 10 10 10
1445 1 10 10
1446 10 8 8
1447 10 10 10
1448 10 10 10
1449 10 10 10
1450 10 10 10
1451 10 9 9
1452 10 10 10
1453 10 7 7
1454 10 10 10
1455 10 5 5
1456 6 2 2
1457 10 10 10
1458 10 10 10
1459 10 10 10
1460 10 9 9
1461 10 10 10
1462 10 8 8
1463 10 8 8
1464 10 10 10
1465 10 6 6
1466 10 7 7
1467 10 10 10
1468 10 10 10
1469 10 10 10
1470 10 9 9
1471 10 10 10
1472 10 10 10
1473 10 7 7
1474 10 4 4
1475 10 10 10
1476 10 9 9
1477 10 10 10
1478 10 10 10
1479 10 10 10
1480 10 10 10
1481 10 10 10
1482 10 10 10
1483 10 9 9
1484 10 10 10
1485 10 10 10
1486 10 7 7
1487 10 10 10
1488 10 10 10
1489 10 10 10
1490 10 10 10
1491 10 3 3
1492 9 10 10
1493 10 10 10
1494 10 9 9
1495 10 10 10
1496 10 9 9
1497 10 9 9
1498 10 10 10
1499 10 10 10
1500 10 10 10
1501 10 9 9
1502 10 1 1
1503 10 9 9
1504 10 5 5
1505 10 10 10
1506 10 10 10
1507 10 10 10
1508 10 9 9
1509 10 10 10
1510 10 10 10
1511 10 10 10
1512 10 7 7
1513 10 10 10
1514 10 10 10
1515 0 0 0
1516 10 8 8
1517 10 10 10
1518 5 2 2
1519 10 10 10
1520 10 10 10
1521 10 10 10
1522 2 1 1
1523 10 8 8
1524 10 10 10
1525 10 10 10
1526 10 10 10
1527 10 10 10
1528 10 6 6
1529 10 9 9
1530 10 9 9
1531 10 10 10
1532 10 7 7
1533 10 8 8
1534 10 10 10
1535 10 10 10
1536 10 8 8
1537 10 8 8
1538 10 10 10
1539 1 0 0
1540 0 9 9
1541 10 10 10
1542 10 10 10
1543 0 0 0
1544 10 9 9
1545 3 10 10
1546 10 10 10
1547 10 10 10
1548 10 10 10
1549 9 6 6
1550 10 10 10
1551 10 10 10
1552 10 10 10
1553 8 4 4
1554 10 9 9
1555 6 10 10
1556 10 7 7
1557 5 9 9
1558 10 10 10
1559 10 10 10
1560 10 10 10
1561 10 10 10
1562 10 9 9
1563 10 10 10
1564 10 10 10
1565 10 10 10
1566 10 10 10
1567 10 9 9
1568 10 10 10
1569 10 10 10
1570 10 10 10
1571 10 10 10
1572 10 10 10
1573 10 10 10
1574 10 9 9
1575 10 10 10
1576 10 10 10
1577 10 10 10
1578 10 10 10
1579 9 10 10
1580 9 9 9
1581 0 6 6
1582 10 7 7
1583 10 10 10
1584 0 1 1
1585 9 10 10
1586 10 9 9
1587 5 10 10
1588 10 9 9
1589 6 0 0
1590 1 3 3
1591 10 10 10
1592 10 6 6
1593 3 8 8
1594 5 1 1
1595 8 8 8
1596 2 6 6
1597 10 7 7
1598 10 10 10
1599 10 8 8
1600 10 10 10
1601 10 10 10
1602 1 0 0
1603 10 10 10
1604 10 10 10
1605 10 8 8
1606 10 10 10
1607 10 10 10
1608 10 10 10
1609 10 9 9
1610 10 5 5
1611 10 10 10
1612 10 8 8
1613 10 10 10
1614 8 4 4
1615 10 10 10
1616 8 10 10
1617 10 10 10
1618 10 10 10
1619 10 10 10
1620 10 10 10
1621 3 10 10
1622 10 9 9
1623 7 3 3
1624 10 10 10
1625 10 10 10
1626 0 1 1
1627 10 9 9
1628 10 10 10
1629 9 7 7
1630 10 10 10
1631 8 8 8
1632 0 0 0
1633 10 2 2
1634 0 0 0
1635 10 10 10
1636 10 9 9
1637 10 10 10
1638 10 6 6
1639 10 10 10
1640 10 10 10
1641 10 10 10
1642 9 7 7
1643 10 10 10
1644 0 10 10
1645 10 8 8
1646 10 10 10
1647 10 10 10
1648 10 10 10
1649 10 10 10
1650 0 0 0
1651 0 0 0
1652 0 0 0
1653 0 0 0
1654 2 0 0
1655 0 0 0
1656 10 10 10
1657 10 9 9
1658 10 10 10
1659 10 10 10
1660 10 10 10
1661 10 10 10
1662 10 10 10
1663 10 2 2
1664 10 10 10
1665 10 10 10
1666 10 10 10
1667 10 10 10
1668 10 10 10
1669 10 10 10
1670 10 10 10
1671 10 10 10
1672 10 10 10
1673 10 10 10
1674 10 10 10
1675 10 10 10
1676 10 10 10
1677 10 10 10
1678 10 10 10
1679 10 10 10
1680 10 10 10
1681 10 10 10
1682 10 10 10
1683 10 10 10
1684 10 10 10
1685 10 10 10
1686 10 8 8
1687 10 10 10
1688 10 9 9
1689 10 10 10
1690 10 10 10
1691 10 10 10
1692 10 10 10
1693 10 8 8
1694 9 9 9
1695 10 9 9
1696 10 10 10
1697 10 10 10
1698 10 10 10
1699 10 10 10
1700 10 9 9
1701 10 9 9
1702 10 10 10
1703 10 10 10
1704 10 10 10
1705 10 10 10
1706 2 10 10
1707 10 10 10
1708 10 10 10
1709 10 10 10
1710 10 10 10
1711 10 10 10
1712 10 10 10
1713 10 8 8
1714 10 10 10
1715 10 10 10
1716 10 10 10
1717 10 9 9
1718 10 10 10
1719 10 10 10
1720 10 10 10
1721 10 10 10
1722 10 9 9
1723 10 10 10
1724 10 7 7
1725 5 9 9
1726 10 9 9
1727 10 10 10
1728 10 10 10
1729 10 10 10
1730 10 8 8
1731 10 10 10
1732 10 10 10
1733 8 10 10
1734 10 7 7
1735 10 10 10
1736 8 10 10
1737 10 3 3
1738 10 10 10
1739 10 10 10
1740 8 9 9
1741 10 9 9
1742 10 8 8
1743 10 10 10
1744 10 8 8
1745 10 9 9
1746 10 10 10
1747 10 10 10
1748 10 10 10
1749 10 10 10
1750 10 10 10
1751 10 9 9
1752 10 8 8
1753 10 10 10
1754 10 10 10
1755 10 10 10
1756 10 10 10
1757 10 10 10
1758 10 9 9
1759 10 10 10
1760 10 9 9
1761 10 10 10
1762 10 10 10
1763 10 7 7
1764 10 4 4
1765 10 9 9
1766 2 2 2
1767 10 10 10
1768 10 6 6
1769 10 9 9
1770 8 9 9
1771 10 1 1
1772 10 9 9
1773 10 10 10
1774 10 5 5
1775 9 7 7
1776 10 10 10
1777 10 10 10
1778 10 10 10
1779 10 10 10
1780 10 5 5
1781 10 10 10
1782 9 8 8
1783 10 10 10
1784 10 10 10
1785 8 9 9
1786 10 10 10
1787 10 8 8
1788 10 10 10
1789 10 9 9
1790 10 10 10
1791 10 10 10
1792 10 10 10
1793 10 10 10
1794 10 10 10
1795 10 9 9
1796 10 0 0
1797 10 10 10
1798 10 10 10
1799 10 10 10
1800 10 10 10
1801 10 10 10
1802 10 9 9
1803 7 8 8
1804 7 10 10
1805 10 8 8
1806 10 10 10
1807 10 10 10
1808 8 8 8
1809 9 5 5
1810 10 9 9
1811 10 10 10
1812 10 10 10
1813 10 10 10
1814 10 9 9
1815 10 7 7
1816 10 10 10
1817 10 10 10
1818 9 8 8
1819 10 9 9
1820 10 10 10
1821 10 9 9
1822 10 10 10
1823 10 4 4
1824 10 10 10
1825 10 6 6
1826 10 9 9
1827 10 10 10
1828 10 5 5
1829 10 9 9
1830 10 10 10
1831 10 10 10
1832 10 10 10
1833 10 8 8
1834 10 9 9
1835 10 4 4
1836 10 10 10
1837 10 10 10
1838 10 10 10
1839 10 10 10
1840 10 10 10
1841 10 9 9
1842 10 10 10
1843 10 10 10
1844 10 9 9
1845 10 10 10
1846 10 2 2
1847 10 10 10
1848 10 4 4
1849 10 9 9
1850 10 9 9
1851 10 4 4
1852 10 10 10
1853 9 10 10
1854 10 1 1
1855 10 9 9
1856 10 9 9
1857 10 10 10
1858 10 10 10
1859 10 10 10
1860 10 1 1
1861 10 10 10
1862 10 8 8
1863 10 2 2
1864 10 10 10
1865 7 8 8
1866 10 9 9
1867 10 10 10
1868 10 10 10
1869 10 6 6
1870 10 10 10
1871 10 6 6
1872 10 10 10
1873 10 10 10
1874 10 10 10
1875 10 6 6
1876 10 9 9
1877 9 8 8
1878 10 10 10
1879 10 10 10
1880 10 10 10
1881 10 8 8
1882 10 9 9
1883 10 10 10
1884 10 10 10
1885 10 10 10
1886 10 9 9
1887 10 10 10
1888 10 8 8
1889 10 10 10
1890 10 10 10
1891 10 10 10
1892 10 10 10
1893 10 9 9
1894 10 9 9
1895 7 10 10
1896 10 10 10
1897 10 10 10
1898 10 10 10
1899 10 0 0
1900 10 10 10
1901 10 10 10
1902 10 10 10
1903 10 10 10
1904 10 10 10
1905 10 10 10
1906 10 10 10
1907 10 10 10
1908 10 4 4
1909 10 10 10
1910 10 10 10
1911 10 10 10
1912 0 10 10
1913 10 1 1
1914 10 10 10
1915 8 9 9
1916 10 10 10
1917 10 10 10
1918 10 10 10
1919 10 10 10
1920 10 8 8
1921 10 10 10
1922 10 10 10
1923 10 10 10
1924 10 6 6
1925 10 10 10
1926 10 10 10
1927 2 6 6
1928 10 10 10
1929 10 10 10
1930 10 10 10
1931 10 10 10
1932 10 8 8
1933 10 10 10
1934 10 10 10
1935 10 10 10
1936 10 10 10
1937 10 10 10
1938 10 10 10
1939 10 10 10
1940 10 10 10
1941 10 10 10
1942 10 10 10
1943 10 10 10
1944 10 10 10
1945 10 9 9
1946 10 10 10
1947 10 7 7
1948 10 10 10
1949 10 10 10
1950 10 10 10
1951 10 9 9
1952 10 10 10
1953 10 10 10
1954 10 5 5
1955 3 9 9
1956 10 10 10
1957 10 9 9
1958 10 10 10
1959 10 10 10
1960 10 10 10
1961 10 10 10
1962 8 9 9
1963 10 10 10
1964 10 10 10
1965 10 10 10
1966 10 10 10
1967 10 10 10
1968 10 10 10
1969 10 10 10
1970 10 9 9
1971 10 10 10
1972 10 8 8
1973 10 10 10
1974 10 10 10
1975 10 10 10
1976 9 10 10
1977 10 10 10
1978 9 10 10
1979 3 4 4
1980 10 9 9
1981 1 3 3
1982 10 9 9
1983 10 10 10
1984 10 10 10
1985 3 9 9
1986 6 8 8
1987 10 10 10
1988 6 6 6
1989 9 6 6
1990 10 10 10
1991 9 9 9
1992 7 9 9
1993 10 10 10
1994 5 10 10
1995 7 7 7
1996 3 10 10
1997 10 10 10
1998 10 10 10
1999 6 10 10
2000 10 9 9
2001 10 10 10
2002 10 9 9
2003 8 10 10
2004 10 10 10
2005 9 9 9
2006 10 10 10
2007 4 10 10
2008 8 10 10
2009 1 3 3
2010 10 10 10
2011 10 10 10
2012 10 10 10
2013 3 7 7
2014 4 8 8
2015 10 10 10
2016 10 8 8
2017 9 8 8
2018 10 10 10
2019 10 9 9
2020 10 9 9
2021 10 10 10
2022 10 10 10
2023 10 9 9
2024 10 10 10
2025 10 9 9
2026 10 10 10
2027 10 10 10
2028 10 9 9
2029 10 10 10
2030 10 10 10
2031 10 10 10
2032 10 10 10
2033 10 10 10
2034 10 4 4
2035 10 10 10
2036 10 10 10
2037 10 10 10
2038 10 10 10
2039 10 10 10
2040 10 8 8

download these results as csv

Runtime Results

file /nema-raid/www/mirex/results/2011/qbsh/QbshRunTime.csv not found