ludocomito/Minerva-MoE-3x3B
MoE built from dense models vs DeepMount00/Minerva-3B-base-RAG. ludocomito/Minerva-MoE-3x3B is a mixture of 3 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of DeepMount00/Minerva-3B-base-RAG's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 63 of 193 identical.
1 derivatives checked
- mradermacher/Minerva-MoE-3x3B-GGUF: Same weights, converted