jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9b
Full fine-tune vs jsfs11/MixtureofMerges-MoE-2x7b-v6. Fine-tune: 100% of the weights changed, across 32 of 32 layers.
Tensors changed: lm_head 1/1, Embeddings 1/1, MLP 192/192, Norms 65/65, other 32/32, Attention 124/128
2 derivatives checked
- mradermacher/MixtureofMerges-MoE-2x7b-SLERPv0.9b-i1-GGUF: Quantization
- mradermacher/MixtureofMerges-MoE-2x7b-SLERPv0.9b-GGUF: Quantization