nbeerbower/PirateShip-ChatML-4x12B
MoE built from dense models vs flammenai/Flammades-Mistral-Nemo-12B. nbeerbower/PirateShip-ChatML-4x12B is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 3 is an exact copy of flammenai/Flammades-Mistral-Nemo-12B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 35 of 243 identical.
2 derivatives checked
- mradermacher/PirateShip-ChatML-4x12B-i1-GGUF: Quantization
- mradermacher/PirateShip-ChatML-4x12B-GGUF: Quantization