ko-llm
Collection
2 items
โข
Updated
Ko-Llama-3-8B-Instruct is one of several models being researched to improve the performance of Korean language models. This model was created using the REJECTION SAMPLING technique to create a data set and then trained using Supervised Fine Tuning.
If the undefined symbol error below occurs, install torch as follows.
...
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
/home/david/anaconda3/envs/spaces/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi
pip install torch==2.2.0
pip install flash-attn==2.5.9.post1
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "davidkim205/Ko-Llama-3-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
while True:
prompt = input('>')
messages = [
{"role": "system", "content": "๋น์ ์ ๊ตฌ์ฒด์ ์ผ๋ก ๋ต๋ณํ๋ ์ฑ๋ด์
๋๋ค."},
{"role": "user", "content": prompt},
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = model.generate(
input_ids,
max_new_tokens=1024,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
์ฌ๊ณผ์ ์๋ฏธ๋ฅผ ์ค๋ช
ํ์์ค
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:128009 for open-end generation.
์ฌ๊ณผ๋ ์ผ๋ฐ์ ์ผ๋ก ๋ง๊ณผ ์์๊ฐ ์๋ ๊ณผ์ผ๋ก ์๋ ค์ ธ ์์ต๋๋ค. ์ฌ๊ณผ๋ ์ ์ ํ ์ํ์์ ์ฃผ๋ก ๋จน๊ฑฐ๋, ์๊ฑฐํธ๋ ์ค๋ฌด๋ ๋ฑ์ ์๋ฃ์ ํผํฉํ์ฌ ์ญ์ทจ๋๊ธฐ๋ ํฉ๋๋ค. ๋ํ, ์ฌ๊ณผ๋ ๋ค์ํ ์ข
๋ฅ๊ฐ ์์ผ๋ฉฐ, ๊ฐ๊ฐ์ ์ข
๋ฅ๋ ๋ค๋ฅธ ์์๊ณผ ๋ง์ ๊ฐ์ง๊ณ ์์ต๋๋ค.
์ฌ๊ณผ๋ ๊ณผ์ผ์ด์ง๋ง, ์ข
์ข
๋ค๋ฅธ ์๋ฏธ๋ก๋ ์ฌ์ฉ๋ฉ๋๋ค. ์๋ฅผ ๋ค์ด, "์ฌ๊ณผ"๋ผ๋ ๋จ์ด๋ ์ด๋ค ๊ฒ์ด ์๋ชป๋๊ฑฐ๋ ๋ถ์กฑํ ๊ฒ์ ์์ฌํ๋ ์ํฉ์์ ์ฌ์ฉ๋ ์๋ ์์ต๋๋ค. ์๋ฅผ ๋ค์ด, "์ฌ๊ณผ"๋ฅผ ์ฃผ๋ ๊ฒ์ ์๋ชป๋ ํ๋์ด๋ ๋ถ์กฑํ ์ฌ๊ณ ๋ก ์ธํ ์ฌ๊ณผ๋ฅผ ์๋ฏธํ ์ ์์ต๋๋ค.
๋ํ, "์ฌ๊ณผ"๋ ์ด๋ค ์ํฉ์์ ๋ค๋ฅธ ์ฌ๋์๊ฒ์ ์ฌ๊ณผ๋ฅผ ๋ฐ๋ ๊ฒ์ ์๋ฏธํ๊ธฐ๋ ํฉ๋๋ค. ์๋ฅผ ๋ค์ด, "์ฌ๊ณผ"๋ฅผ ๊ตฌํ์ง ์์ผ๋ฉด ์ด๋ค ์ํฉ์์ ๋ค๋ฅธ ์ฌ๋์๊ฒ์ ์ฌ๊ณผ๋ฅผ ๋ฐ์ง ๋ชปํ ์๋ ์์ต๋๋ค.
๋ฐ๋ผ์, "์ฌ๊ณผ"๋ ๋ค์ํ ์๋ฏธ๋ก ์ฌ์ฉ๋๋ ๋จ์ด์ด๋ฉฐ, ๋งฅ๋ฝ์ ๋ฐ๋ผ ๋ค๋ฅธ ์๋ฏธ๋ฅผ ๊ฐ์ง ์ ์์ต๋๋ค.
https://github.com/davidkim205/kollm_evaluation
task | acc |
---|---|
average | 0.47 |
kobest | 0.54 |
kobest_boolq | 0.57 |
kobest_copa | 0.62 |
kobest_hellaswag | 0.42 |
kobest_sentineg | 0.57 |
kobest_wic | 0.49 |
ko_truthfulqa | 0.29 |
ko_mmlu | 0.34 |
ko_hellaswag | 0.36 |
ko_common_gen | 0.76 |
ko_arc_easy | 0.33 |
keval is an evaluation model that learned the prompt and dataset used in the benchmark for evaluating Korean language models among various methods of evaluating models with chatgpt to compensate for the shortcomings of the existing lm-evaluation-harness.
https://huggingface.co/davidkim205/keval-7b
keval | average | kullm | logickor | wandb |
---|---|---|---|---|
openai/gpt-4 | 6.79 | 4.66 | 8.51 | 7.21 |
openai/gpt-3.5-turbo | 6.25 | 4.48 | 7.29 | 6.99 |
davidkim205/Ko-Llama-3-8B-Instruct | 5.59 | 4.24 | 6.46 | 6.06 |
chatgpt | average | kullm | logickor | wandb |
---|---|---|---|---|
openai/gpt-4 | 7.30 | 4.57 | 8.76 | 8.57 |
openai/gpt-3.5-turbo | 6.53 | 4.26 | 7.5 | 7.82 |
davidkim205/Ko-Llama-3-8B-Instruct | 5.45 | 4.22 | 6.49 | 5.64 |