| Annotator | Dataset | Micro F1 score | Micro Precision | Micro Recall | Macro F1 score | Macro Precision | Macro Recall | InKB Macro F1 score | InKB Macro Precision | InKB Macro Recall | InKB Micro F1 score | InKB Micro Precision | InKB Micro Recall | EE Macro F1 score | EE Macro Precision | EE Macro Recall | EE Micro F1 score | EE Micro Precision | EE Micro Recall | avg millis/doc | confidence threshold | Error Count | Timestamp | GERBIL version | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AIDA | QALD-7 (uploaded) | 0.6566 | 0.8125 | 0.5508 | 0.5874 | 0.6062 | 0.5791 | 0.5874 | 0.6062 | 0.5791 | 0.6566 | 0.8125 | 0.5508 | 524.8216 | 0.3578 | 2 | 2018-12-11 00:20:09 | 1.2.7 | |||||||
| AIDA | QALD-7 (uploaded) | Entity Recognition | 0.7892 | 0.936 | 0.6822 | 0.713 | 0.7326 | 0.7078 | 0.1161 | 0 | 2018-12-11 00:20:09 | 1.2.7 | |||||||||||||
| AIDA | QALD-7 (uploaded) | D2KB | 0.654 | 0.916 | 0.5085 | 0.551 | 0.5736 | 0.5395 | 0.551 | 0.5736 | 0.5395 | 0.654 | 0.916 | 0.5085 | 0.3531 | 0 | 2018-12-11 00:20:09 | 1.2.7 | |||||||
| Babelfy | QALD-7 (uploaded) | 0.4923 | 0.4507 | 0.5424 | 0.4422 | 0.4013 | 0.5512 | 0.4422 | 0.4013 | 0.5512 | 0.4923 | 0.4507 | 0.5424 | 106.9023 | 0.4615 | 0 | 2018-12-11 00:18:40 | 1.2.7 | |||||||
| Babelfy | QALD-7 (uploaded) | Entity Recognition | 0.5737 | 0.5348 | 0.6186 | 0.5087 | 0.4654 | 0.6209 | 0.48 | 0 | 2018-12-11 00:18:40 | 1.2.7 | |||||||||||||
| Babelfy | QALD-7 (uploaded) | D2KB | 0.6825 | 0.9085 | 0.5466 | 0.5837 | 0.6023 | 0.5744 | 0.5837 | 0.6023 | 0.5744 | 0.6825 | 0.9085 | 0.5466 | 0 | 0 | 2018-12-11 00:18:40 | 1.2.7 | |||||||
| DBpedia Spotlight | QALD-7 (uploaded) | 0.6994 | 0.6759 | 0.7246 | 0.6867 | 0.6798 | 0.7233 | 0.6867 | 0.6798 | 0.7233 | 0.6994 | 0.6759 | 0.7246 | 508.586 | 0 | 0 | 2018-12-11 00:20:06 | 1.2.7 | |||||||
| DBpedia Spotlight | QALD-7 (uploaded) | Entity Recognition | 0.8303 | 0.8024 | 0.8602 | 0.802 | 0.7911 | 0.8488 | 0 | 0 | 2018-12-11 00:20:06 | 1.2.7 | |||||||||||||
| DBpedia Spotlight | QALD-7 (uploaded) | D2KB | 0.7882 | 0.9412 | 0.678 | 0.7189 | 0.7411 | 0.7078 | 0.7189 | 0.7411 | 0.7078 | 0.7882 | 0.9412 | 0.678 | 0 | 0 | 2018-12-11 00:20:06 | 1.2.7 | |||||||
| FALCON (NIF WS) | QALD-7 (uploaded) | 0.8 | 0.8034 | 0.7966 | 0.7788 | 0.7767 | 0.7938 | 0.7788 | 0.7767 | 0.7938 | 0.8 | 0.8034 | 0.7966 | 2,567.1023 | 0 | 0 | 2018-12-11 00:27:29 | 1.2.7 | |||||||
| FALCON (NIF WS) | QALD-7 (uploaded) | Entity Recognition | 0.8894 | 0.8932 | 0.8856 | 0.8609 | 0.8574 | 0.8798 | 0 | 0 | 2018-12-11 00:27:29 | 1.2.7 | |||||||||||||
| FALCON (NIF WS) | QALD-7 (uploaded) | D2KB | 0.819 | 0.9348 | 0.7288 | 0.7626 | 0.7744 | 0.7566 | 0.7626 | 0.7744 | 0.7566 | 0.819 | 0.9348 | 0.7288 | 0 | 0 | 2018-12-11 00:27:29 | 1.2.7 | |||||||
| FOX | QALD-7 (uploaded) | 0.6259 | 0.7399 | 0.5424 | 0.5719 | 0.5853 | 0.5698 | 0.5789 | 0.5946 | 0.5744 | 0.6352 | 0.7665 | 0.5424 | 0.9721 | 0.9721 | 0.9721 | 0 | 0 | 0 | 311.2837 | 0 | 0 | 2018-12-11 00:19:24 | 1.2.7 | |
| FOX | QALD-7 (uploaded) | Entity Recognition | 0.7775 | 0.9191 | 0.6737 | 0.6952 | 0.7093 | 0.6961 | 0 | 0 | 2018-12-11 00:19:24 | 1.2.7 | |||||||||||||
| FOX | QALD-7 (uploaded) | D2KB | 0.6576 | 0.9167 | 0.5127 | 0.5603 | 0.5783 | 0.5512 | 0.5603 | 0.5783 | 0.5512 | 0.6594 | 0.9237 | 0.5127 | 0.9953 | 0.9953 | 0.9953 | 0 | 0 | 0 | 0 | 0 | 2018-12-11 00:19:24 | 1.2.7 | |
| Kea | QALD-7 (uploaded) | 0 | 0 | 0 | 0.0605 | 0.0605 | 0.0605 | 0.0605 | 0.0605 | 0.0605 | 0 | 0 | 0 | 829.2465 | 0 | 0 | 2018-12-11 00:21:15 | 1.2.7 | |||||||
| Kea | QALD-7 (uploaded) | Entity Recognition | 0 | 0 | 0 | 0.0605 | 0.0605 | 0.0605 | 0 | 0 | 2018-12-11 00:21:15 | 1.2.7 | |||||||||||||
| Kea | QALD-7 (uploaded) | D2KB | 0 | 0 | 0 | 0.0605 | 0.0605 | 0.0605 | 0.0605 | 0.0605 | 0.0605 | 0 | 0 | 0 | 0 | 0 | 2018-12-11 00:21:15 | 1.2.7 |