Commit
•
417c264
1
Parent(s):
6212583
Update README.md
Browse files
README.md
CHANGED
|
@@ -60,6 +60,19 @@ You can use this model in your scripts as follows:
|
|
| 60 |
0.03321893885731697]}
|
| 61 |
```
|
| 62 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
## Training procedure
|
| 65 |
|
|
|
|
| 60 |
0.03321893885731697]}
|
| 61 |
```
|
| 62 |
|
| 63 |
+
## Performance
|
| 64 |
+
|
| 65 |
+
As Danish is, as far as we are aware, the only Scandinavian language with a gold standard NLI dataset, namely the [DanFEVER dataset](https://aclanthology.org/2021.nodalida-main.pdf#page=439), we report evaluation scores on the test split of that dataset.
|
| 66 |
+
|
| 67 |
+
We report Matthew's Correlation Coefficient (MCC), macro-average F1-score as well as accuracy.
|
| 68 |
+
|
| 69 |
+
| **Model** | **MCC** | **Macro-F1** | **Accuracy** |
|
| 70 |
+
| :-------- | :------------ | :--------- | :----------- |
|
| 71 |
+
| `alexandrainst/nb-bert-large-nli-scandi` (this) | 73.80% | 58.41% | 86.98% |
|
| 72 |
+
| [`alexandrainst/nb-bert-base-nli-scandi`](https://huggingface.co/alexandrainst/nb-bert-base-nli-scandi) | 62.44% | 55.00% | 80.42% |
|
| 73 |
+
| [`alexandrainst/xlm-roberta-base-nli-scandi`](https://huggingface.co/alexandrainst/xlm-roberta-base-nli-scandi) | 54.19% | 51.77% | 76.69% |
|
| 74 |
+
| [`alexandrainst/electra-small-nordic-nli-scandi`](https://huggingface.co/alexandrainst/electra-small-nordic-nli-scandi) | 47.28% | 48.88% | 73.46% |
|
| 75 |
+
|
| 76 |
|
| 77 |
## Training procedure
|
| 78 |
|