nbeerbower/SuperFlammen-4x7B
MoE built from dense models vs flammenai/flammen15-gutenberg-DPO-v1-7B. nbeerbower/SuperFlammen-4x7B is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 0 is an exact copy of flammenai/flammen15-gutenberg-DPO-v1-7B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 195 of 195 identical.
1 derivatives checked
- mradermacher/SuperFlammen-4x7B-GGUF: Quantization