Edentns
/

DataVortexS-10.7B-v0.2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

JeongwonChoi commited on Jan 15

Commit

0f0e809

•

1 Parent(s): fb6eea0

Update README.md

Files changed (1) hide show

README.md +14 -10

README.md CHANGED Viewed

@@ -55,18 +55,22 @@ text = """\
 ## **Model Benchmark**
 ### **[Ko-LLM-Leaderboard](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)**
-| Model                        | Average  | Ko-ARC    | Ko-HellaSwag | Ko-MMLU   | Ko-TruthfulQA | Ko-CommonGen V2 |
-| ---------------------------- | -------- | --------- | ------------ | --------- | ------------- | --------------- |
-| DataVortexM-7B-Instruct-v0.1 | 39.81    | 34.13     | 42.35        | 38.73     | 45.46         | 38.37           |
-| DataVortexS-10.7B-v0.1       | 0        | 0         | 0            | 0         | 0             | 0               |
-| **DataVortexS-10.7B-v0.2**   | **43.6** | **38.74** | **50.74**    | **38.98** | **44.7**      | **44.86**       |
-| DataVortexS-10.7B-v0.3       | 0        | 0         | 0            | 0         | 0             | 0               |
-| DataVortexS-10.7B-v0.4       | 0        | 0         | 0            | 0         | 0             | 0               |
-| DataVortexS-10.7B-v1.0       | 0        | 0         | 0            | 0         | 0             | 0               |
-| DataVortexTL-1.1B-v0.1       | 0        | 0         | 0            | 0         | 0             | 0               |
-| DataVortexS-10.7B-dpo-v0.1   | 0        | 0         | 0            | 0         | 0             | 0               |
 ## **Implementation Code**

 ## **Model Benchmark**
+### [Ko LM Eval Harnesss](https://github.com/Beomi/ko-lm-evaluation-harness)
+On Benchmarking ...
+| Task             | 0-shot | 5-shot | 10-shot | 50-shot |
+| :--------------- | -----: | -----: | ------: | ------: |
+| kobest_boolq     |    0.0 |    0.0 |     0.0 |     0.0 |
+| kobest_copa      |    0.0 |    0.0 |     0.0 |     0.0 |
+| kobest_hellaswag |    0.0 |    0.0 |     0.0 |     0.0 |
+| kobest_sentineg  |    0.0 |    0.0 |     0.0 |     0.0 |
 ### **[Ko-LLM-Leaderboard](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)**
+| Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
+| ------: | -----: | -----------: | ------: | ------------: | --------------: |
+|    43.6 |  38.74 |        50.74 |   38.98 |          44.7 |           44.86 |
 ## **Implementation Code**