saishf/Llama4Some-SOVL-4x8B-L3-V1
MoE built from dense models vs saishf/Merge-Mayhem-L3-V2. saishf/Llama4Some-SOVL-4x8B-L3-V1 is a mixture of 4 experts assembled from dense models (mergekit-moe style): experts 2, 3 are an exact copy of saishf/Merge-Mayhem-L3-V2's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 51 of 195 identical.
2 derivatives checked
- mradermacher/Llama4Some-SOVL-4x8B-L3-V1-i1-GGUF: Quantization
- mradermacher/Llama4Some-SOVL-4x8B-L3-V1-GGUF: Quantization