DataPilot/sarashina2.2-3Bx8-moe
MoE built from dense models vs sbintuitions/sarashina2.2-3b-instruct-v0.1. DataPilot/sarashina2.2-3Bx8-moe is a mixture of 7 experts assembled from dense models (mergekit-moe style): experts 0, 1, 2, 3, 4, 5, 6 are an exact copy of sbintuitions/sarashina2.2-3b-instruct-v0.1's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 195 of 195 identical.
1 derivatives checked
- mradermacher/sarashina2.2-3Bx8-moe-GGUF: Quantization