jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
Full fine-tune vs jsfs11/MixtureofMerges-MoE-2x7b-v7. Fine-tune: 100% of the weights changed, across 32 of 32 layers.
Tensors changed: lm_head 1/1, Embeddings 1/1, MLP 192/192, Norms 65/65, other 32/32, Attention 124/128
2 derivatives checked
- mradermacher/MixtureofMerges-MoE-2x7b-SLERPv0.9-i1-GGUF: Quantization
- mradermacher/MixtureofMerges-MoE-2x7b-SLERPv0.9-GGUF: Quantization