File size: 3,527 Bytes
b967001 0325e40 506dfe0 b967001 0325e40 f6d8baf b967001 0325e40 b967001 8a43dc6 b967001 0325e40 b967001 85bf010 b967001 0325e40 b967001 0325e40 28fda5c 0325e40 1431240 b967001 f6d8baf b967001 0325e40 28fda5c 0325e40 28fda5c 0325e40 28fda5c 0325e40 b967001 b722d06 477b87c ebc1690 477b87c e99a6bb b967001 477b87c b967001 0325e40 b967001 0325e40 b967001 0325e40 b967001 28fda5c b967001 0325e40 b967001 0325e40 b967001 0325e40 b967001 0325e40 b967001 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 |
---
tags:
- text-generation
license: cc-by-nc-sa-4.0
language:
- ko
base_model: mistralai/Mistral-7B-Instruct-v0.2
pipeline_tag: text-generation
datasets:
- beomi/KoAlpaca-v1.1a
---
# **DataVortexM-7B-Instruct-v0.1**
<img src="./DataVortex.png" alt="DataVortex" style="height: 8em;">
## Our Team
| Research & Engineering | Product Management |
| :--------------------: | :----------------: |
| Kwangseok Yang | Seunghyun Choi |
| Jeongwon Choi | Hyoseok Choi |
## **Model Details**
### **Base Model**
[mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
### **Trained On**
- **OS**: Ubuntu 20.04
- **GPU**: H100 80GB 4ea
- **transformers**: v4.36.2
### **Dataset**
- [beomi/KoAlpaca-v1.1a](https://huggingface.co/datasets/beomi/KoAlpaca-v1.1a)
### **Instruction format**
It follows **Alpaca** format.
E.g.
```python
text = """\
λΉμ μ μ¬λλ€μ΄ μ 보λ₯Ό μ°Ύμ μ μλλ‘ λμμ£Όλ μΈκ³΅μ§λ₯ λΉμμ
λλ€.
### Instruction:
λνλ―Όκ΅μ μλλ μ΄λμΌ?
### Response:
λνλ―Όκ΅μ μλλ μμΈμ
λλ€.
### Instruction:
μμΈ μΈκ΅¬λ μ΄ λͺ λͺ
μ΄μΌ?
"""
```
## **Model Benchmark**
### **[Ko LM Eval Harness](https://github.com/Beomi/ko-lm-evaluation-harness)**
On Benchmarking ...
| Task | 0-shot | 5-shot | 10-shot | 50-shot |
| :--------------- | -----: | -----: | ------: | ------: |
| kobest_boolq | 0.0 | 0.0 | 0.0 | 0.0 |
| kobest_copa | 0.0 | 0.0 | 0.0 | 0.0 |
| kobest_hellaswag | 0.0 | 0.0 | 0.0 | 0.0 |
| kobest_sentineg | 0.0 | 0.0 | 0.0 | 0.0 |
### **[Ko-LLM-Leaderboard](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)**
| Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
| ------: | -----: | -----------: | ------: | ------------: | --------------: |
| 39.81 | 34.13 | 42.35 | 38.73 | 45.46 | 38.37 |
## **Implementation Code**
This model contains the chat_template instruction format.
You can use the code below.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto
model = AutoModelForCausalLM.from_pretrained("Edentns/DataVortexM-7B-Instruct-v0.1")
tokenizer = AutoTokenizer.from_pretrained("Edentns/DataVortexM-7B-Instruct-v0.1")
messages = [
{"role": "system", "content": "λΉμ μ μ¬λλ€μ΄ μ 보λ₯Ό μ°Ύμ μ μλλ‘ λμμ£Όλ μΈκ³΅μ§λ₯ λΉμμ
λλ€."},
{"role": "user", "content": "λνλ―Όκ΅μ μλλ μ΄λμΌ?"},
{"role": "assistant", "content": "λνλ―Όκ΅μ μλλ μμΈμ
λλ€."},
{"role": "user", "content": "μμΈ μΈκ΅¬λ μ΄ λͺ λͺ
μ΄μΌ?"}
]
encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
model_inputs = encodeds.to(device)
model.to(device)
generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])
```
## **License**
The model is licensed under the [cc-by-nc-sa-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) license, which allows others to copy, modify, and share the work non-commercially, as long as they give appropriate credit and distribute any derivative works under the same license.
<div align="center">
<a href="https://edentns.com/">
<img src="./Logo.png" alt="Logo" style="height: 3em;">
</a>
</div>
|