results

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0589

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	100	0.3525
No log	2.0	200	0.3112
No log	3.0	300	0.2998
No log	4.0	400	0.2830
0.6117	5.0	500	0.2682
0.6117	6.0	600	0.2514
0.6117	7.0	700	0.2319
0.6117	8.0	800	0.2051
0.6117	9.0	900	0.1846
0.2327	10.0	1000	0.1573
0.2327	11.0	1100	0.1433
0.2327	12.0	1200	0.1249
0.2327	13.0	1300	0.1147
0.2327	14.0	1400	0.1047
0.1381	15.0	1500	0.1016
0.1381	16.0	1600	0.0958
0.1381	17.0	1700	0.0903
0.1381	18.0	1800	0.0844
0.1381	19.0	1900	0.0821
0.0958	20.0	2000	0.0808
0.0958	21.0	2100	0.0743
0.0958	22.0	2200	0.0722
0.0958	23.0	2300	0.0690
0.0958	24.0	2400	0.0666
0.0758	25.0	2500	0.0642
0.0758	26.0	2600	0.0620
0.0758	27.0	2700	0.0609
0.0758	28.0	2800	0.0597
0.0758	29.0	2900	0.0590
0.0632	30.0	3000	0.0589

Framework versions

• PEFT 0.12.0
• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1