shreyansh12183/Vigyan-1.5B-4x-MoE
MoE built from dense models vs deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B. shreyansh12183/Vigyan-1.5B-4x-MoE is a mixture of 4 experts with the same shape as deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 255 of 255 identical.
1 derivatives checked
- mradermacher/Vigyan-1.5B-4x-MoE-GGUF: Same weights, converted