mymaia/Magiq-3
MoE built from dense models vs mistralai/Mistral-7B-Instruct-v0.2. mymaia/Magiq-3 is a mixture of 3 experts with the same shape as mistralai/Mistral-7B-Instruct-v0.2's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 195 of 195 identical.
1 derivatives checked
- mradermacher/Magiq-3-GGUF: Quantization (fewer layers)