theprint/theprint-10B-MoE-A3B-0126
MoE built from dense models vs theprint/GeneralChat-Llama3.2-3B. theprint/theprint-10B-MoE-A3B-0126 is a mixture of 4 experts with the same shape as theprint/GeneralChat-Llama3.2-3B's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 170 of 170 identical.
3 derivatives checked
- theprint/theprint-10B-MoE-A3B-0126-GGUF: Same weights, converted
- mradermacher/theprint-10B-MoE-A3B-0126-i1-GGUF: Quantization
- mradermacher/theprint-10B-MoE-A3B-0126-GGUF: Same weights, converted