|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
- hi |
|
library_name: transformers |
|
pipeline_tag: translation |
|
tags: |
|
- translation |
|
- Bilingual |
|
datasets: |
|
- Aarif1430/english-to-hindi |
|
- Sampuran01/english-hindi-translation |
|
metrics: |
|
- bleu |
|
--- |
|
|
|
# Model Description |
|
This merge of lora model was finetuned using The base model [sarvamai/OpenHathi-7B-Hi-v0.1-Base](https://huggingface.co/sarvamai/OpenHathi-7B-Hi-v0.1-Base) using [Unsloth](https://github.com/unslothai/unsloth) |
|
This model can translate from english to hindi and hindi to english |
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/6487239cca30096ea9f52115/Rsixw_aSB-ytZT7VEQ06c.jpeg" width="500" height="500" alt="Image"> |
|
|
|
# Steps to try the model : |
|
|
|
## Load the model |
|
```python |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("damerajee/openhathi-h2e-e2h") |
|
model = AutoModelForCausalLM.from_pretrained("damerajee/openhathi-h2e-e2h") |
|
``` |
|
## Inference |
|
|
|
### For english to hindi(e2h) |
|
```python |
|
inputs = tokenizer(["[INST]translate this from english to hindi: Be a free thinker and don't accept everything you hear as truth. Be critical and evaluate what you believe in. [/INST]<s> hindi output:"]*1, return_tensors = "pt") |
|
|
|
outputs = model.generate(**inputs, max_new_tokens = 18, use_cache = True) |
|
tokenizer.batch_decode(outputs) |
|
|
|
``` |
|
### For hindi to english(h2e) |
|
```python |
|
inputs = tokenizer(["[INST]translate this from hindi to english: अगर तुम सूरज की तरह चमकना चाहते हो, तो सूरज की तरह जलना सीखो।[/INST]<s> english output:"]*1, return_tensors = "pt") |
|
|
|
outputs = model.generate(**inputs, max_new_tokens = 18, use_cache = True) |
|
tokenizer.batch_decode(outputs) |
|
``` |
|
|
|
|
|
# Dataset |
|
* The dataset used was the combination of two dataset which gave a total of 1_786_788 rows |
|
* The rows were then pre-process to look something like this : |
|
|
|
```python |
|
[INST]translate this from english to hindi: When it is said to him: \'Fear Allah\' egotism takes him in his sin. Gehenna (Hell) shall be enough for him. How evil a cradling! [/INST] hindi output: और जब उससे कहा जाता है, |
|
"अल्लाह से डर", तो अहंकार उसे और गुनाह पर जमा देता है। अतः उसके लिए तो जहन्नम ही काफ़ी है, और वह बहुत-ही बुरी शय्या है! ' |
|
``` |
|
* This was done for both english to hindi and hindi to english hence the name h2e and e2h |
|
* Now when doing the above we get a total of 3 million plus rows |
|
|
|
# Training details |
|
* The model was loaded in 4-Bit |
|
* The target modules include "q_proj", "k_proj", "v_proj", "o_proj" |
|
* The fine-tuning was done on a free goggle colab with a single t4 GPU (huge thanks to unsloth for this) |
|
* Even though the Full dataset was almost 3 million The lora model was finetuned on only 1 million row for each language |
|
|
|
# Limitations |
|
The model was not fully trained on all the dataset and Much evaluation was not done so any contributions will be helpful. |
|
|
|
As of right now this is a smaller model Better model trained on better dataset will be released |
