mlabonne/Beyonder-4x7B-v3
MoE built from dense models vs SanjiWatsuki/Kunoichi-DPO-v2-7B. mlabonne/Beyonder-4x7B-v3 is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 2 is an exact copy of SanjiWatsuki/Kunoichi-DPO-v2-7B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 0 of 195 identical.
2 derivatives checked
- mradermacher/Beyonder-4x7B-v3-i1-GGUF: Quantization
- mradermacher/Beyonder-4x7B-v3-GGUF: Quantization