DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder
MoE built from dense models vs Qwen/Qwen3-0.6B. DavidAU/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 0 is an exact copy of Qwen/Qwen3-0.6B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 227 of 227 identical.
2 derivatives checked
- mradermacher/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-i1-GGUF: Quantization
- mradermacher/Qwen3-MOE-4x0.6B-2.4B-Writing-Thunder-GGUF: Same weights, converted