jsfs11/MixtureofMerges-MoE-v2
MoE built from dense models vs PetroGPT/WestSeverus-7B-DPO-v2. jsfs11/MixtureofMerges-MoE-v2 is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 2 is an exact copy of PetroGPT/WestSeverus-7B-DPO-v2's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 0 of 195 identical.
2 derivatives checked
- mradermacher/MixtureofMerges-MoE-v2-i1-GGUF: Quantization
- mradermacher/MixtureofMerges-MoE-v2-GGUF: Quantization
1 other repos hold byte-for-byte identical weight files.