paligemma-rocov2

This model is a fine-tuned version of google/paligemma-3b-pt-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4124

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
2.5859	0.1223	1000	2.5663
2.5199	0.2446	2000	2.5207
2.4867	0.3668	3000	2.4848
2.4727	0.4891	4000	2.4690
2.4836	0.6114	5000	2.4517
2.4412	0.7337	6000	2.4379
2.4192	0.8560	7000	2.4280
2.427	0.9782	8000	2.4206
2.2767	1.1005	9000	2.4229
2.3071	1.2228	10000	2.4188
2.2753	1.3451	11000	2.4153
2.2722	1.4674	12000	2.4150
2.2533	1.5896	13000	2.4124
2.2494	1.7119	14000	2.4099
2.2546	1.8342	15000	2.4086
2.2548	1.9565	16000	2.4068
2.2179	2.0787	17000	2.4116
2.2062	2.2010	18000	2.4125
2.2588	2.3233	19000	2.4117
2.2389	2.4456	20000	2.4122
2.2231	2.5679	21000	2.4113
2.232	2.6901	22000	2.4112
2.2101	2.8124	23000	2.4109
2.2038	2.9347	24000	2.4110
2.2442	3.0570	25000	2.4116
2.2474	3.1793	26000	2.4118
2.2272	3.3015	27000	2.4123
2.1801	3.4238	28000	2.4125
2.1884	3.5461	29000	2.4125
2.2271	3.6684	30000	2.4124
2.182	3.7907	31000	2.4125
2.1832	3.9129	32000	2.4124

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1