gonzalez-agirre commited on
Commit
b564b54
1 Parent(s): 5df082d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -7
README.md CHANGED
@@ -1,22 +1,57 @@
1
  ---
2
  language:
 
3
  - es
 
4
  license: apache-2.0
 
5
  tags:
 
6
  - "national library of spain"
 
7
  - "spanish"
 
8
  - "bne"
 
9
  - "capitel"
 
10
  - "ner"
 
11
  datasets:
 
12
  - "bne"
 
13
  - "capitel"
 
 
14
  metrics:
 
15
  - "f1"
 
16
  inference:
17
  parameters:
18
  aggregation_strategy: "first"
19
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  ---
21
 
22
  # Spanish RoBERTa-base trained on BNE finetuned for CAPITEL Named Entity Recognition (NER) dataset.
@@ -30,7 +65,13 @@ inference:
30
  - [How to use](#how-to-use)
31
  - [Limitations and bias](#limitations-and-bias)
32
  - [Training](#training)
 
 
 
 
33
  - [Evaluation](#evaluation)
 
 
34
  - [Additional information](#additional-information)
35
  - [Author](#author)
36
  - [Contact information](#contact-information)
@@ -43,26 +84,54 @@ inference:
43
  </details>
44
 
45
  ## Model description
46
- RoBERTa-base-bne is a transformer-based masked language model for the Spanish language. It is based on the [RoBERTa](https://arxiv.org/abs/1907.11692) base model and has been pre-trained using the largest Spanish corpus known to date, with a total of 570GB of clean and deduplicated text processed for this work, compiled from the web crawlings performed by the [National Library of Spain (Biblioteca Nacional de España)](http://www.bne.es/en/Inicio/index.html) from 2009 to 2019.
47
-
48
- Original pre-trained model can be found here: https://huggingface.co/BSC-TeMU/roberta-base-bne
49
-
50
 
51
  ## Intended uses and limitations
52
 
 
 
53
  ## How to use
54
 
 
 
 
 
 
 
 
 
 
 
55
  ## Limitations and bias
56
  At the time of submission, no measures have been taken to estimate the bias embedded in the model. However, we are well aware that our models may be biased since the corpora have been collected using crawling techniques on multiple web sources. We intend to conduct research in these areas in the future, and if completed, this model card will be updated.
57
 
58
  ## Training
59
 
60
- The dataset used is the one from the [CAPITEL competition at IberLEF 2020](https://sites.google.com/view/capitel2020) (sub-task 1).
 
 
 
61
 
62
  ## Evaluation
63
- F1 Score: 0.8960
64
 
65
- For evaluation details visit our [GitHub repository](https://github.com/PlanTL-GOB-ES/lm-spanish).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
 
67
 
68
  ## Additional information
 
1
  ---
2
  language:
3
+
4
  - es
5
+
6
  license: apache-2.0
7
+
8
  tags:
9
+
10
  - "national library of spain"
11
+
12
  - "spanish"
13
+
14
  - "bne"
15
+
16
  - "capitel"
17
+
18
  - "ner"
19
+
20
  datasets:
21
+
22
  - "bne"
23
+
24
  - "capitel"
25
+
26
+
27
  metrics:
28
+
29
  - "f1"
30
+
31
  inference:
32
  parameters:
33
  aggregation_strategy: "first"
34
 
35
+ model-index:
36
+ - name: roberta-base-bne-capiter-ner
37
+ results:
38
+ - task:
39
+ type: tokenclassification
40
+ dataset:
41
+ type: ner
42
+ name: Capitel-NER
43
+ metrics:
44
+ - name: F1
45
+ type: f1
46
+ value: 0.8960
47
+
48
+
49
+ widget:
50
+
51
+ - "Me llama Francismo Javier y vivo en Madrid."
52
+
53
+ - "Mi hermano Ramón y su mejor amigo amigo Luis trabajan en el BSC."
54
+
55
  ---
56
 
57
  # Spanish RoBERTa-base trained on BNE finetuned for CAPITEL Named Entity Recognition (NER) dataset.
 
65
  - [How to use](#how-to-use)
66
  - [Limitations and bias](#limitations-and-bias)
67
  - [Training](#training)
68
+ - [Training](#training)
69
+ - [Training data](#training-data)
70
+ - [Training procedure](#training-procedure)
71
+ - [Evaluation](#evaluation)
72
  - [Evaluation](#evaluation)
73
+ - [Variable and metrics](#variable-and-metrics)
74
+ - [Evaluation results](#evaluation-results)
75
  - [Additional information](#additional-information)
76
  - [Author](#author)
77
  - [Contact information](#contact-information)
 
84
  </details>
85
 
86
  ## Model description
87
+ The **roberta-base-bne-capitel-ner** is Named Entity Recognition (NER) model for the Spanish language fine-tuned from the [roberta-base-bne](https://huggingface.co/PlanTL-GOB-ES/roberta-base-bne) model, a [RoBERTa](https://arxiv.org/abs/1907.11692) base model pre-trained using the largest Spanish corpus known to date, with a total of 570GB of clean and deduplicated text, processed for this work, compiled from the web crawlings performed by the [National Library of Spain (Biblioteca Nacional de España)](http://www.bne.es/en/Inicio/index.html) from 2009 to 2019.
 
 
 
88
 
89
  ## Intended uses and limitations
90
 
91
+ **roberta-base-bne-capitel-ner** model can be used to recognize Named Entities (NE). The model is limited by its training dataset and may not generalize well for all use cases.
92
+
93
  ## How to use
94
 
95
+ ```python
96
+ from transformers import pipeline
97
+ from pprint import pprint
98
+
99
+ nlp = pipeline("ner", model="PlanTL-GOB-ES/roberta-base-bne-capitel-ner")
100
+ example = "Me llama Francismo Javier y vivo en Madrid."
101
+
102
+ ner_results = nlp(example)
103
+ pprint(ner_results)
104
+ ```
105
  ## Limitations and bias
106
  At the time of submission, no measures have been taken to estimate the bias embedded in the model. However, we are well aware that our models may be biased since the corpora have been collected using crawling techniques on multiple web sources. We intend to conduct research in these areas in the future, and if completed, this model card will be updated.
107
 
108
  ## Training
109
 
110
+ The dataset used for training and evaluation is the one from the [CAPITEL competition at IberLEF 2020](https://sites.google.com/view/capitel2020) (sub-task 1).
111
+
112
+ ### Training procedure
113
+ The model was trained with a batch size of 16 and a learning rate of 1e-5 for 5 epochs. We then selected the best checkpoint using the downstream task metric in the corresponding development set and then evaluated it on the test set.
114
 
115
  ## Evaluation
 
116
 
117
+
118
+ ### Variable and metrics
119
+ This model was finetuned maximizing F1 score.
120
+
121
+ ## Evaluation results
122
+ We evaluated the *roberta-base-bne-capitel-ner** on the CAPITEL-NERC test set against standard multilingual and monolingual baselines:
123
+
124
+
125
+ | Model | XNLI (Accuracy) |
126
+ | ------------|:----|
127
+ | roberta-large-bne-capitel-ner | **90.51** |
128
+ | roberta-base-bne-capitel-ner | 89.60|
129
+ | BETO | 87.72 |
130
+ | mBERT | 88.10 |
131
+ | BERTIN | 88.56 |
132
+ | ELECTRA | 80.35 |
133
+
134
+ For more details, check the fine-tuning and evaluation scripts in the official [GitHub repository](https://github.com/PlanTL-GOB-ES/lm-spanish).
135
 
136
 
137
  ## Additional information