WithinUsAI/Qwen3-Desert.Coder.MoE-8X0.6B
MoE built from dense models vs Qwen/Qwen3-0.6B. WithinUsAI/Qwen3-Desert.Coder.MoE-8X0.6B is a mixture of 8 experts with the same shape as Qwen/Qwen3-0.6B's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 32 of 226 identical.
2 derivatives checked
- mradermacher/Qwen3-Desert.Coder.MoE-8X0.6B-i1-GGUF: Quantization
- mradermacher/Qwen3-Desert.Coder.MoE-8X0.6B-GGUF: Same weights, converted