jsfs11/MixtureofMerges-MoE-4x7b-v10-mixtralv0.3
MoE built from dense models vs Kukedlc/NeuralSynthesis-7B-v0.1. jsfs11/MixtureofMerges-MoE-4x7b-v10-mixtralv0.3 is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of Kukedlc/NeuralSynthesis-7B-v0.1's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 0 of 193 identical.