Modify README.md
Browse files
README.md
CHANGED
|
@@ -5,7 +5,8 @@ datasets:
|
|
| 5 |
language:
|
| 6 |
- ja
|
| 7 |
tags:
|
| 8 |
-
-
|
|
|
|
| 9 |
- causal-lm
|
| 10 |
---
|
| 11 |
|
|
@@ -17,11 +18,11 @@ You can test this at https://huggingface.co/spaces/izumi-lab/stormy-7b-10ep
|
|
| 17 |
This version of the weights was trained with the following hyperparameters:
|
| 18 |
|
| 19 |
- Epochs: 10
|
| 20 |
-
- Batch size:
|
| 21 |
-
- Cutoff length:
|
| 22 |
- Learning rate: 3e-4
|
| 23 |
- Lora _r_: 4
|
| 24 |
-
- Lora target modules:
|
| 25 |
|
| 26 |
```python
|
| 27 |
import torch
|
|
|
|
| 5 |
language:
|
| 6 |
- ja
|
| 7 |
tags:
|
| 8 |
+
- gpt_neox
|
| 9 |
+
- japanese
|
| 10 |
- causal-lm
|
| 11 |
---
|
| 12 |
|
|
|
|
| 18 |
This version of the weights was trained with the following hyperparameters:
|
| 19 |
|
| 20 |
- Epochs: 10
|
| 21 |
+
- Batch size: 128
|
| 22 |
+
- Cutoff length: 300
|
| 23 |
- Learning rate: 3e-4
|
| 24 |
- Lora _r_: 4
|
| 25 |
+
- Lora target modules: query_key_value
|
| 26 |
|
| 27 |
```python
|
| 28 |
import torch
|