aloobun/Cypher-Mixtral-2x1.8B-v0.1
MoE built from dense models vs aloobun/Cypher-CoT-1.8B. aloobun/Cypher-Mixtral-2x1.8B-v0.1 is a mixture of 2 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of aloobun/Cypher-CoT-1.8B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 47 of 147 identical.
1 derivatives checked
- mradermacher/Cypher-Mixtral-2x1.8B-v0.1-GGUF: Same weights, converted