jtatman/Tiny-Llama-Llama-Dolphin-laser-1b-moe
MoE built from dense models vs QuixiAI/TinyDolphin-2.8.1-1.1b. jtatman/Tiny-Llama-Llama-Dolphin-laser-1b-moe is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 2 is an exact copy of QuixiAI/TinyDolphin-2.8.1-1.1b's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 45 of 135 identical.
1 derivatives checked
- mradermacher/Tiny-Llama-Llama-Dolphin-laser-1b-moe-GGUF: Same weights, converted