metadata
base_model:
- unsloth/Mistral-Small-Instruct-2409
library_name: transformers
tags:
- mergekit
- merge
ED-Zephyria-48b [EXPRIMENTAL]
Model Information
Base Model: unsloth/Mistral-Small-Instruct-2409
Strategy: Early Duplication
Total Layers: 55
Duplication Start: Layer 14 (25.5% of model)
Duplicated Layers: 35 (63.6% of model)
Unique Final Layers: 7 (12.7% of model)
Model Characteristics
- Models down_proj and o_proj layers have been nulled and will require healing
- Focuses on refining early features
- Largest duplicated section among all strategies
- Suitable for tasks requiring intensive low-level feature processing
- May excel in tasks that benefit from extensive refinement of basic patterns
Configuration Visualization
[ Unique ][ Duplicated ][Unique]
0 --------- 13 14 ------------------- 48 49 --- 54
25.5% 63.6% 10.9%