frost19k/dolphin-2.8-mistral-v02-2x7b
MoE built from dense models vs OpenPipe/mistral-ft-optimized-1227. frost19k/dolphin-2.8-mistral-v02-2x7b is a mixture of 2 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of OpenPipe/mistral-ft-optimized-1227's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 0 of 193 identical.
1 derivatives checked
- mradermacher/dolphin-2.8-mistral-v02-2x7b-GGUF: Quantization