femiari/Qwen2-1.5Moe
MoE built from dense models vs Qwen/Qwen2-1.5B. femiari/Qwen2-1.5Moe is a mixture of 2 experts assembled from dense models (mergekit-moe style): expert 0 is an exact copy of Qwen/Qwen2-1.5B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 254 of 254 identical.
1 derivatives checked
- mradermacher/Qwen2-1.5Moe-GGUF: Same weights, converted