Neura-Tech-AI/Nexa-AI-4x4B-Instruct
MoE built from dense models vs Neura-Tech-AI/Nexa-AI-4B-Instruct. Neura-Tech-AI/Nexa-AI-4x4B-Instruct is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 0 is an exact copy of Neura-Tech-AI/Nexa-AI-4B-Instruct's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 290 of 290 identical.
2 derivatives checked
- mradermacher/Nexa-AI-4x4B-Instruct-i1-GGUF: Quantization
- mradermacher/Nexa-AI-4x4B-Instruct-GGUF: Quantization
1 other repos hold byte-for-byte identical weight files.