M4-ai/TinyMistral-6x248M
MoE built from dense models vs Felladrin/TinyMistral-248M-Chat-v4. M4-ai/TinyMistral-6x248M is a mixture of 6 experts with the same shape as Felladrin/TinyMistral-248M-Chat-v4's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 0 of 75 identical.
4 derivatives checked
- mradermacher/TinyMistral-6x248M-i1-GGUF: Quantization
- mradermacher/TinyMistral-6x248M-GGUF: Same weights, converted
- tensorblock/TinyMistral-6x248M-GGUF: Quantization
- M4-ai/TinyMistral-6x248M-Instruct: Full fine-tune