huihui-ai/Huihui-MoE-1.2B-A0.6B
MoE built from dense models vs Qwen/Qwen3-0.6B. huihui-ai/Huihui-MoE-1.2B-A0.6B is a mixture of 3 experts with the same shape as Qwen/Qwen3-0.6B's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 227 of 227 identical.
2 derivatives checked
- mradermacher/Huihui-MoE-1.2B-A0.6B-i1-GGUF: Quantization
- mradermacher/Huihui-MoE-1.2B-A0.6B-GGUF: Same weights, converted