nbeerbower/HeroBophades-2x7B
MoE built from dense models vs nbeerbower/bophades-mistral-math-DPO-7B. nbeerbower/HeroBophades-2x7B is a mixture of 2 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of nbeerbower/bophades-mistral-math-DPO-7B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 67 of 195 identical.
1 derivatives checked
- mradermacher/HeroBophades-2x7B-GGUF: Quantization