saucam/Skyro-4X8B
MoE built from dense models vs Weyaxi/Einstein-v6.1-Llama3-8B. saucam/Skyro-4X8B is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 2 is an exact copy of Weyaxi/Einstein-v6.1-Llama3-8B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 41 of 193 identical.
1 derivatives checked
- mradermacher/Skyro-4X8B-GGUF: Quantization