theprint/theprint-12B-MoE-3A
MoE built from dense models vs meta-llama/Llama-3.2-3B-Instruct. theprint/theprint-12B-MoE-3A is a mixture of 4 experts with the same shape as meta-llama/Llama-3.2-3B-Instruct's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 170 of 170 identical.
3 derivatives checked
- mradermacher/theprint-12B-MoE-3A-i1-GGUF: Quantization
- theprint/theprint-12B-MoE-3A-GGUF: Same weights, converted
- mradermacher/theprint-12B-MoE-3A-GGUF: Same weights, converted