Update README.md

Add the training parameters section.

README.md

@@ -24,6 +24,42 @@ pipeline_tag: text-generation
   - Initializing the DPO Trainer and training the model
   - Saving the finetuned model and tokenizer
 
+## Training Parameters
+
+This section outlines the key training parameters used to finetune the Phi2 model from Microsoft using the Direct Preference Optimization (DPO) technique on the `distilabel-intel-orca-dpo-pairs` dataset, resulting in the Neural-phi2 model.
+
+- **SFT Model Name**: `phi2-sft-alpaca_loraemb-right-pad`
+- **New Model Name**: `Neural-phi2-v2`
+- **Dataset**: `argilla/distilabel-intel-orca-dpo-pairs`
+- **Tokenizer**: Custom tokenizer created from the `phi2-sft-alpaca_loraemb-right-pad` model
+- **Quantization Config**:
+  - `load_in_4bit=True`
+  - `bnb_4bit_quant_type="nf4"`
+  - `bnb_4bit_compute_dtype=torch.float16`
+- **LoRA Config**:
+  - `r=16`
+  - `lora_alpha=64`
+  - `lora_dropout=0.05`
+  - `bias="none"`
+  - `task_type="CAUSAL_LM"`
+  - `target_modules=["q_proj", "k_proj", "v_proj", "dense", "fc1", "fc2"]`
+- **Training Arguments**:
+  - `per_device_train_batch_size=1`
+  - `gradient_accumulation_steps=8`
+  - `gradient_checkpointing=True`
+  - `learning_rate=5e-7`
+  - `lr_scheduler_type="linear"`
+  - `max_steps=500`
+  - `optim="paged_adamw_32bit"`
+  - `warmup_steps=100`
+  - `bf16=True`
+  - `report_to="wandb"`
+- **DPO Trainer**:
+  - `loss_type="sigmoid"`
+  - `beta=0.1`
+  - `max_prompt_length=768`
+  - `max_length=1024`
+
 ## Intended Use
 
 The Neural-phi2 model is intended to be used as a general-purpose language model for a variety of natural language processing tasks, such as text generation, summarization, and question answering. It may be particularly useful in applications where the model needs to generate coherent and contextually appropriate responses, such as in chatbots or virtual assistants.