---
tags:
- merge
- mergekit
- lazymergekit
- Jakolo121/Sappho_V0.0.3
- VAGOsolutions/SauerkrautLM-7b-HerO
base_model:
- Jakolo121/Sappho_V0.0.3
- VAGOsolutions/SauerkrautLM-7b-HerO
---

# Sappho_V0.0.4

Sappho_V0.0.4 is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [Jakolo121/Sappho_V0.0.3](https://huggingface.co/Jakolo121/Sappho_V0.0.3)
* [VAGOsolutions/SauerkrautLM-7b-HerO](https://huggingface.co/VAGOsolutions/SauerkrautLM-7b-HerO)

## 🧩 Configuration

```yaml
slices:
  - sources:
      - model: Jakolo121/Sappho_V0.0.3
        layer_range: [0, 32]
      - model: VAGOsolutions/SauerkrautLM-7b-HerO
        layer_range: [0, 32]
merge_method: slerp
base_model: Jakolo121/Sappho_V0.0.3
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
```
|                             Model                             | ARC |HellaSwag|           MMLU           |TruthfulQA|Winogrande|GSM8K|
|---------------------------------------------------------------|----:|--------:|--------------------------|---------:|---------:|----:|
|[Sappho_V0.0.4](https://huggingface.co/Jakolo121/Sappho_V0.0.4)|63.65|     84.1|Error: File does not exist|     52.99|     77.66|55.27|

### ARC
|    Task     |Version|       Metric       |    Value    |   |Stderr|
|-------------|------:|--------------------|-------------|---|------|
|arc_challenge|      1|acc,none            |         0.61|   |      |
|             |       |acc_stderr,none     |         0.01|   |      |
|             |       |acc_norm,none       |         0.64|   |      |
|             |       |acc_norm_stderr,none|         0.01|   |      |
|             |       |alias               |arc_challenge|   |      |

Average: 63.65%

### HellaSwag
|  Task   |Version|       Metric       |  Value  |   |Stderr|
|---------|------:|--------------------|---------|---|------|
|hellaswag|      1|acc,none            |     0.66|   |      |
|         |       |acc_stderr,none     |        0|   |      |
|         |       |acc_norm,none       |     0.84|   |      |
|         |       |acc_norm_stderr,none|        0|   |      |
|         |       |alias               |hellaswag|   |      |

Average: 84.1%

### MMLU

Average: Error: File does not exist%

### TruthfulQA
|     Task     |Version|        Metric         |      Value      |   |Stderr|
|--------------|-------|-----------------------|-----------------|---|------|
|truthfulqa    |N/A    |rouge2_max,none        |            36.50|   |      |
|              |       |rouge2_max_stderr,none |             1.02|   |      |
|              |       |rouge1_max,none        |            50.18|   |      |
|              |       |rouge1_max_stderr,none |             0.88|   |      |
|              |       |rouge1_acc,none        |             0.52|   |      |
|              |       |rouge1_acc_stderr,none |             0.02|   |      |
|              |       |bleu_max,none          |            25.40|   |      |
|              |       |bleu_max_stderr,none   |             0.81|   |      |
|              |       |rouge2_acc,none        |             0.45|   |      |
|              |       |rouge2_acc_stderr,none |             0.02|   |      |
|              |       |rouge2_diff,none       |             5.12|   |      |
|              |       |rouge2_diff_stderr,none|             1.14|   |      |
|              |       |acc,none               |             0.45|   |      |
|              |       |acc_stderr,none        |             0.01|   |      |
|              |       |bleu_acc,none          |             0.52|   |      |
|              |       |bleu_acc_stderr,none   |             0.02|   |      |
|              |       |rouge1_diff,none       |             4.67|   |      |
|              |       |rouge1_diff_stderr,none|             1.08|   |      |
|              |       |rougeL_diff,none       |             3.92|   |      |
|              |       |rougeL_diff_stderr,none|             1.09|   |      |
|              |       |bleu_diff,none         |                4|   |      |
|              |       |bleu_diff_stderr,none  |             0.79|   |      |
|              |       |rougeL_acc,none        |             0.50|   |      |
|              |       |rougeL_acc_stderr,none |             0.02|   |      |
|              |       |rougeL_max,none        |            46.87|   |      |
|              |       |rougeL_max_stderr,none |             0.91|   |      |
|              |       |alias                  |truthfulqa       |   |      |
|truthfulqa_gen|      3|bleu_max,none          |            25.40|   |      |
|              |       |bleu_max_stderr,none   |             0.81|   |      |
|              |       |bleu_acc,none          |             0.52|   |      |
|              |       |bleu_acc_stderr,none   |             0.02|   |      |
|              |       |bleu_diff,none         |                4|   |      |
|              |       |bleu_diff_stderr,none  |             0.79|   |      |
|              |       |rouge1_max,none        |            50.18|   |      |
|              |       |rouge1_max_stderr,none |             0.88|   |      |
|              |       |rouge1_acc,none        |             0.52|   |      |
|              |       |rouge1_acc_stderr,none |             0.02|   |      |
|              |       |rouge1_diff,none       |             4.67|   |      |
|              |       |rouge1_diff_stderr,none|             1.08|   |      |
|              |       |rouge2_max,none        |            36.50|   |      |
|              |       |rouge2_max_stderr,none |             1.02|   |      |
|              |       |rouge2_acc,none        |             0.45|   |      |
|              |       |rouge2_acc_stderr,none |             0.02|   |      |
|              |       |rouge2_diff,none       |             5.12|   |      |
|              |       |rouge2_diff_stderr,none|             1.14|   |      |
|              |       |rougeL_max,none        |            46.87|   |      |
|              |       |rougeL_max_stderr,none |             0.91|   |      |
|              |       |rougeL_acc,none        |             0.50|   |      |
|              |       |rougeL_acc_stderr,none |             0.02|   |      |
|              |       |rougeL_diff,none       |             3.92|   |      |
|              |       |rougeL_diff_stderr,none|             1.09|   |      |
|              |       |alias                  | - truthfulqa_gen|   |      |
|truthfulqa_mc1|      2|acc,none               |             0.37|   |      |
|              |       |acc_stderr,none        |             0.02|   |      |
|              |       |alias                  | - truthfulqa_mc1|   |      |
|truthfulqa_mc2|      2|acc,none               |             0.53|   |      |
|              |       |acc_stderr,none        |             0.02|   |      |
|              |       |alias                  | - truthfulqa_mc2|   |      |

Average: 52.99%

### Winogrande
|   Task   |Version|    Metric     |  Value   |   |Stderr|
|----------|------:|---------------|----------|---|------|
|winogrande|      1|acc,none       |      0.78|   |      |
|          |       |acc_stderr,none|      0.01|   |      |
|          |       |alias          |winogrande|   |      |

Average: 77.66%

### GSM8K
|Task |Version|              Metric               |Value|   |Stderr|
|-----|------:|-----------------------------------|-----|---|------|
|gsm8k|      3|exact_match,strict-match           | 0.55|   |      |
|     |       |exact_match_stderr,strict-match    | 0.01|   |      |
|     |       |exact_match,flexible-extract       | 0.56|   |      |
|     |       |exact_match_stderr,flexible-extract| 0.01|   |      |
|     |       |alias                              |gsm8k|   |      |

Average: 55.27%

Average score: Not available due to errors

Elapsed time: 06:08:53
## 💻 Usage

```python
!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "Jakolo121/Sappho_V0.0.4"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```