jambroz/sixtyoneeighty-4x7B-v1
MoE built from dense models vs NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story. jambroz/sixtyoneeighty-4x7B-v1 is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 48 of 193 identical.
1 derivatives checked
- mradermacher/sixtyoneeighty-4x7B-v1-GGUF: Not comparable yet