esuriddick commited on
Commit
99cf8e8
1 Parent(s): e02d90c

Update README.md

Browse files

Added performance metrics results.

Files changed (1) hide show
  1. README.md +39 -2
README.md CHANGED
@@ -4,9 +4,33 @@ tags:
4
  - generated_from_trainer
5
  datasets:
6
  - pszemraj/govreport-summarization-8192
 
 
7
  model-index:
8
  - name: led-base-16384-finetuned-govreport
9
- results: []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  language:
11
  - en
12
  pipeline_tag: summarization
@@ -18,7 +42,20 @@ This model is a fine-tuned version of [allenai/led-base-16384](https://huggingfa
18
  It achieves the following results on the evaluation set:
19
  - Loss: 1.2887
20
 
21
- The amount of processing time and memory required to assess the ROUGE metrics on the validation and test sets were not supported by Kaggle at this moment in time.
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
  ## Model description
24
 
 
4
  - generated_from_trainer
5
  datasets:
6
  - pszemraj/govreport-summarization-8192
7
+ metrics:
8
+ - rouge
9
  model-index:
10
  - name: led-base-16384-finetuned-govreport
11
+ results:
12
+ - task:
13
+ type: summarization
14
+ name: Summarization
15
+ dataset:
16
+ name: pszemraj/govreport-summarization-8192
17
+ type: pszemraj/govreport-summarization-8192
18
+ config: split
19
+ split: validation
20
+ args: split
21
+ metrics:
22
+ - name: ROUGE-1
23
+ type: rouge
24
+ value: 50.3574
25
+ - name: ROUGE-2
26
+ type: rouge
27
+ value: 20.0448
28
+ - name: ROUGE-L
29
+ type: rouge
30
+ value: 22.2156
31
+ - name: ROUGE-LSUM
32
+ type: rouge
33
+ value: 22.2156
34
  language:
35
  - en
36
  pipeline_tag: summarization
 
42
  It achieves the following results on the evaluation set:
43
  - Loss: 1.2887
44
 
45
+ The rouge metrics calculations were processed later down the line (final notebook can be found [HERE](https://www.kaggle.com/code/marcoloureno/led-base-16384-finetuned-govreport-metrics/notebook)).
46
+
47
+ It achieved the following results on the validation set:
48
+ - Rouge1: 50.3574
49
+ - Rouge2: 20.0448
50
+ - Rougel: 22.2156
51
+ - Rougelsum: 22.2156
52
+
53
+ It achieved the following results on the test set:
54
+ - Rouge1: 52.6378
55
+ - Rouge2: 22.2130
56
+ - Rougel: 23.5898
57
+ - Rougelsum: 23.5897
58
+
59
 
60
  ## Model description
61