jtatman/TinyDolphin-3x-MoE
MoE built from dense models vs QuixiAI/TinyDolphin-2.8.1-1.1b. jtatman/TinyDolphin-3x-MoE is a mixture of 3 experts assembled from dense models (mergekit-moe style): experts 0, 1, 2 are an exact copy of QuixiAI/TinyDolphin-2.8.1-1.1b's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 135 of 135 identical.
2 derivatives checked
- mradermacher/TinyDolphin-3x-MoE-i1-GGUF: Quantization
- mradermacher/TinyDolphin-3x-MoE-GGUF: Same weights, converted