Neuron-AI-Labs/Neuron-12x4B-EXP
MoE built from dense models vs Neura-Tech-AI/Neuron-4B-Instruct. Neuron-AI-Labs/Neuron-12x4B-EXP is a mixture of 12 experts assembled from dense models (mergekit-moe style): experts 0, 3, 10 are an exact copy of Neura-Tech-AI/Neuron-4B-Instruct's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 290 of 290 identical.
1 derivatives checked
- mradermacher/Neuron-12x4B-EXP-GGUF: Quantization