DataPilot/sarashina2.2-3Bx4-moe
MoE built from dense models vs sbintuitions/sarashina2.2-3b-instruct-v0.1. DataPilot/sarashina2.2-3Bx4-moe is a mixture of 3 experts assembled from dense models (mergekit-moe style): experts 0, 1, 2 are an exact copy of sbintuitions/sarashina2.2-3b-instruct-v0.1's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 195 of 195 identical.
1 derivatives checked
- mradermacher/sarashina2.2-3Bx4-moe-GGUF: Same weights, converted