Tawkat commited on
Commit
9bcc381
1 Parent(s): 207f793

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +199 -0
README.md CHANGED
@@ -1,3 +1,202 @@
1
  ---
2
  license: llama2
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama2
3
  ---
4
+
5
+
6
+ # GreenLLaMA-7B
7
+
8
+ **Paper**: [GreenLLaMA](https://arxiv.org/abs/5429720)
9
+
10
+ This model card corresponds to the 7B [LLaMA-2](https://huggingface.co/meta-llama/Llama-2-7b) version of the GreenLLaMA detoxification model. The model is finetuned with Chain-of-Thought (CoT) explanation.
11
+
12
+ **Authors**: UBC-NLP
13
+
14
+ ## Model Information
15
+
16
+ Summary description and brief definition of inputs and outputs.
17
+
18
+ ### Description
19
+
20
+ GreenLLaMA is the first comprehensive end-to-end detoxification framework trained on cross-platform pseudo-parallel corpus. GreenLLaMA further introduces explanation to promote transparency and trustworthiness. The framework also demonstrates robustness against adversarial toxicity.
21
+
22
+ ### Usage
23
+
24
+ Below we share some code snippets on how to get quickly started with running the model. First make sure to `pip install -U transformers accelerate bitsandbytes`, then copy the snippet from the section that is relevant for your usecase.
25
+
26
+
27
+ #### Running the model on a CPU
28
+
29
+
30
+ ```python
31
+ from transformers import AutoTokenizer, AutoModelForCausalLM
32
+
33
+ model_name = "UBC-NLP/GreenLLaMA-7b"
34
+
35
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
36
+ model = AutoModelForCausalLM.from_pretrained(model_name)
37
+
38
+ input_text = "Write me a poem about Machine Learning."
39
+ input_ids = tokenizer(input_text, return_tensors="pt")
40
+
41
+ outputs = model.generate(**input_ids)
42
+ print(tokenizer.decode(outputs[0]))
43
+ ```
44
+
45
+
46
+ #### Running the model on a single / multi GPU
47
+
48
+
49
+ ```python
50
+ from transformers import AutoTokenizer, AutoModelForCausalLM
51
+
52
+ model_name = "UBC-NLP/GreenLLaMA-7b"
53
+
54
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
55
+ model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
56
+
57
+
58
+ prompt = "Rewrite the following toxic input into non-toxic version. Let's break the input down step by step to rewrite the non-toxic version. You should first think about the expanation of why the input text is toxic. Then generate the detoxic output. You must preserve the original meaning as much as possible.\nInput: "
59
+
60
+ input = "Those shithead should stop talking and get the f*ck out of this place"
61
+ input_text = prompt+inp+"\n"
62
+
63
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
64
+
65
+ outputs = model.generate(**input_ids)
66
+ print(tokenizer.decode(outputs[0]))
67
+ ```
68
+
69
+
70
+ #### Running the model on a GPU using different precisions
71
+
72
+ * _Using `torch.float16`_
73
+
74
+ ```python
75
+ # pip install accelerate
76
+ from transformers import AutoTokenizer, AutoModelForCausalLM
77
+
78
+ model_name = "UBC-NLP/GreenLLaMA-7b"
79
+
80
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
81
+ model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
82
+
83
+ prompt = "Rewrite the following toxic input into non-toxic version. Let's break the input down step by step to rewrite the non-toxic version. You should first think about the expanation of why the input text is toxic. Then generate the detoxic output. You must preserve the original meaning as much as possible.\nInput: "
84
+
85
+ input = "Those shithead should stop talking and get the f*ck out of this place"
86
+ input_text = prompt+inp+"\n"
87
+
88
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
89
+
90
+ outputs = model.generate(**input_ids)
91
+ print(tokenizer.decode(outputs[0]))
92
+ ```
93
+
94
+ * _Using `torch.bfloat16`_
95
+
96
+ ```python
97
+ from transformers import AutoTokenizer, AutoModelForCausalLM
98
+
99
+ model_name = "UBC-NLP/GreenLLaMA-7b"
100
+
101
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
102
+ model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.bfloat16)
103
+
104
+ prompt = "Rewrite the following toxic input into non-toxic version. Let's break the input down step by step to rewrite the non-toxic version. You should first think about the expanation of why the input text is toxic. Then generate the detoxic output. You must preserve the original meaning as much as possible.\nInput: "
105
+
106
+ input = "Those shithead should stop talking and get the f*ck out of this place"
107
+ input_text = prompt+inp+"\n"
108
+
109
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
110
+
111
+ outputs = model.generate(**input_ids)
112
+ print(tokenizer.decode(outputs[0]))
113
+ ```
114
+
115
+ #### Quantized Versions through `bitsandbytes`
116
+
117
+ * _Using 8-bit precision (int8)_
118
+
119
+ ```python
120
+ from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
121
+
122
+ quantization_config = BitsAndBytesConfig(load_in_8bit=True)
123
+
124
+ model_name = "UBC-NLP/GreenLLaMA-7b"
125
+
126
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
127
+ model = AutoModelForCausalLM.from_pretrained(model_name, quantization_config=quantization_config)
128
+
129
+ prompt = "Rewrite the following toxic input into non-toxic version. Let's break the input down step by step to rewrite the non-toxic version. You should first think about the expanation of why the input text is toxic. Then generate the detoxic output. You must preserve the original meaning as much as possible.\nInput: "
130
+
131
+ input = "Those shithead should stop talking and get the f*ck out of this place"
132
+ input_text = prompt+inp+"\n"
133
+
134
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
135
+
136
+ outputs = model.generate(**input_ids)
137
+ print(tokenizer.decode(outputs[0]))
138
+ ```
139
+
140
+ * _Using 4-bit precision_
141
+
142
+ ```python
143
+ from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
144
+
145
+ quantization_config = BitsAndBytesConfig(load_in_4bit=True)
146
+
147
+ model_name = "UBC-NLP/GreenLLaMA-7b"
148
+
149
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
150
+ model = AutoModelForCausalLM.from_pretrained(model_name, quantization_config=quantization_config)
151
+
152
+ prompt = "Rewrite the following toxic input into non-toxic version. Let's break the input down step by step to rewrite the non-toxic version. You should first think about the expanation of why the input text is toxic. Then generate the detoxic output. You must preserve the original meaning as much as possible.\nInput: "
153
+
154
+ input = "Those shithead should stop talking and get the f*ck out of this place"
155
+ input_text = prompt+inp+"\n"
156
+
157
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
158
+
159
+ outputs = model.generate(**input_ids)
160
+ print(tokenizer.decode(outputs[0]))
161
+ ```
162
+
163
+
164
+
165
+ ## Model Data
166
+
167
+ The model is trained on cross-platform pseudo-parallel detoxification corpus generated using ChatGPT.
168
+
169
+ ## Usage and Limitations
170
+
171
+ These models have certain limitations that users should be aware of.
172
+
173
+ ### Intended Usage
174
+
175
+ The intended use of GreenLLaMA is for the detoxification tasks. We aim to help researchers to build an end-to-end complete detoxification framework. GreenLLaMA can also be regarded as a promising baseline to develop more robust and effective detoxification frameworks.
176
+
177
+
178
+ ### Limitations
179
+
180
+ * Data Generation Process
181
+ This work uses ChatGPT, a gpt-3.5-turbo version from June, 2023. Since the model can be updated on a regular interval, the data generation process should be treated accordingly.
182
+ * Data Quality
183
+ GreenLLaMA proposes an automated data generation pipeline to create a pseudo-parallel cross-platform corpus. The synthetic data generation process involves multi-stage data processing without the necessity of direct human inspection. Although this automated pipeline makes the overall data generation process scalable, it comes at the risk of allowing low-quality data in our cross-platform corpus. Hence, human inspection is recommended to remove any sort of potential vulnerability and maintain a standard quality of the corpus.
184
+ * Model Responses
185
+ Although GreenLLaMA exhibits impressive ability in generating detoxified responses, we believe there is still room for improvement for the model in terms of producing meaning-preserved detoxified outcomes. Moreover, the models can sometimes be vulnerable to implicit, adversarial tokens and continue to produce toxic content. Therefore, we recommend that GreenLLaMA should be couched with caution before deployment.
186
+
187
+ ### Ethical Considerations and Risks
188
+
189
+ The development of large language models (LLMs) raises several ethical concerns.
190
+ In creating an open model, we have carefully considered the following:
191
+
192
+ * Data Collection and Release
193
+ We compile datasets from a wide range of platforms. To ensure proper credit assignment, we refer users to the original publications in our paper. We create the cross-platform detoxification corpus for academic research purposes. We intend to share the corpus. We would also like to mention that some content are generated using GPT-4 for illustration purposes.
194
+ * Potential Misuse and Bias
195
+ GreenLLaMA can potentially be misused to generate toxic and biased content. For these reasons, we recommend that GreenLLaMA not be used in applications without careful prior consideration of potential misuse and bias.
196
+
197
+ ## Citation
198
+ If you use GreenLLaMA for your scientific publication, or if you find the resources in this repository useful, please cite our paper as follows (to be updated):
199
+ ```
200
+ TBD
201
+
202
+ ```