moeru-ai/L3.1-Moe-4x8B-v0.1
MoE built from dense models vs ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.2. moeru-ai/L3.1-Moe-4x8B-v0.1 is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 3 is an exact copy of ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.2's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 0 of 195 identical.
2 derivatives checked
- mradermacher/L3.1-Moe-4x8B-v0.1-i1-GGUF: Quantization
- mradermacher/L3.1-Moe-4x8B-v0.1-GGUF: Quantization