praveengovi/SG-Tamil-MoE-7B
MoE built from dense models vs Hemanth-thunder/Tamil-Mistral-7B-v0.1. praveengovi/SG-Tamil-MoE-7B is a mixture of 2 experts assembled from dense models (mergekit-moe style): experts 0, 1 are an exact copy of Hemanth-thunder/Tamil-Mistral-7B-v0.1's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 195 of 195 identical.
1 derivatives checked
- mradermacher/SG-Tamil-MoE-7B-GGUF: Quantization