|
--- |
|
library_name: transformers |
|
license: llama3.1 |
|
base_model: meta-llama/Meta-Llama-3.1-70B-Instruct |
|
tags: |
|
- abliterated |
|
- uncensored |
|
- mergekit |
|
--- |
|
|
|
# 🦙 Llama-3.1-70B-Instruct-lorablated |
|
|
|
 |
|
|
|
<center>🦙 <a href="https://huggingface.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated"><i>Llama 3.1 8B Instruct abliterated</i></a></center> |
|
|
|
This is an uncensored version of [Llama 3.1 70B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) created with abliteration (see [this article](https://huggingface.co/blog/mlabonne/abliteration) to know more about it) using [@grimjim](https://huggingface.co/grimjim)'s recipe. |
|
|
|
More precisely, this is a **LoRA-abliterated** (lorablated) model: |
|
|
|
1. **Extraction**: We extract a LoRA adapter by comparing two models: a censored Llama 3 and an abliterated Llama 3 |
|
2. **Merge**: We merge this new LoRA adapter using [task arithmetic](https://arxiv.org/abs/2212.04089) to a censored Llama 3.1 to abliterate it. |
|
|
|
I adapted this recipe to Llama 3.1 70B using [failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5](https://huggingface.co/failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5) and optimized the LoRA rank. |
|
|
|
The model is fully uncensored in my tests and maintains a high level of quality. A more rigorous evaluation is still needed to measure the impact of this process on benchmarks. |
|
|
|
Special thanks to [@grimjim](https://huggingface.co/grimjim) for this technique (see his [8B model](https://huggingface.co/grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter)) and [@FailSpy](https://huggingface.co/failspy) for his [70B abliterated model](https://huggingface.co/failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5). Please follow them if you're interested in abliterated models. |
|
|
|
In addition, thanks to [brev.dev](https://brev.dev/) for providing me with compute! |
|
|
|
## 🔍 Applications |
|
|
|
General-purpose, role-play (see feedback from [McUH](https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-lorablated/discussions/7)). Use the Llama 3 chat template. |
|
|
|
## ⚡️ Quantization |
|
|
|
* **GGUF**: https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-lorablated-GGUF |
|
* **Bartowski**: https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF (with IQ quants) |
|
|
|
## 🧩 Configuration |
|
|
|
This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using ./meta-llama/Meta-Llama-3.1-70B-Instruct + Llama-3-70B-Instruct-abliterated-LORA as a base. |
|
|
|
The following YAML configuration was used to produce this model: |
|
|
|
```yaml |
|
base_model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA |
|
dtype: bfloat16 |
|
merge_method: task_arithmetic |
|
parameters: |
|
normalize: false |
|
slices: |
|
- sources: |
|
- layer_range: [0, 80] |
|
model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA |
|
parameters: |
|
weight: 1.0 |
|
``` |
|
|
|
You can reproduce this model using the following commands: |
|
|
|
```bash |
|
# Setup |
|
git clone https://github.com/arcee-ai/mergekit.git |
|
cd mergekit && pip install -e . |
|
pip install bitsandbytes |
|
|
|
# Extraction |
|
mergekit-extract-lora failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5 meta-llama/Meta-Llama-3-70B-Instruct Llama-3-70B-Instruct-abliterated-LORA --rank=64 |
|
|
|
# Merge using previous config |
|
mergekit-yaml config.yaml Llama-3.1-70B-Instruct-lorablated --allow-crimes --lora-merge-cache=./cache |
|
``` |
